Is Learning in Biological Neural Networks Based on Stochastic Gradient Descent? An Analysis Using Stochastic Processes.

Publisher: MIT Press Country of Publication: United States NLM ID: 9426182 Publication Model: Print Cited Medium: Internet ISSN: 1530-888X (Electronic) Linking ISSN: 08997667 NLM ISO Abbreviation: Neural Comput Subsets: MEDLINE

Original Publication: Cambridge, Mass. : MIT Press, c1989-

Neural Networks, Computer* ; Stochastic Processes* ; Supervised Machine Learning* ; Nerve Net*/cytology ; Nerve Net*/physiology ; Models, Neurological*; Deep Learning

In recent years, there has been an intense debate about how learning in biological neural networks (BNNs) differs from learning in artificial neural networks. It is often argued that the updating of connections in the brain relies only on local information, and therefore a stochastic gradient-descent type optimization method cannot be used. In this note, we study a stochastic model for supervised learning in BNNs. We show that a (continuous) gradient step occurs approximately when each learning opportunity is processed by many local updates. This result suggests that stochastic gradient descent may indeed play a role in optimizing BNNs.
(© 2024 Massachusetts Institute of Technology.)

Date Created: 20240426 Date Completed: 20240630 Latest Revision: 20240630

20250114

10.1162/neco_a_01668

38669690