“The Hebbian rule is a very narrow, particular and not very sensitive way of using error information,” said Daniel Yamins, a computational neuroscientist and computer scientist at Stanford University.
#Backpropagation in neural networks update#
This principle, with some modifications, was successful at explaining certain limited types of learning and visual classification tasks.īut it worked far less well for large networks of neurons that had to learn from mistakes there was no directly targeted way for neurons deep within the network to learn about discovered errors, update themselves and make fewer mistakes. “But it also has value in its own right.” Learning Through Backpropagationįor decades, neuroscientists’ theories about how brains learn were guided primarily by a rule introduced in 1949 by the Canadian psychologist Donald Hebb, which is often paraphrased as “Neurons that fire together, wire together.” That is, the more correlated the activity of adjacent neurons, the stronger the synaptic connections between them. There’s a general impression that if we can unlock some of its principles, it might be helpful for AI,” said Bengio. All these efforts are bringing us closer to understanding the algorithms that may be at work in the brain. Some researchers are also incorporating the properties of certain types of cortical neurons and processes such as attention into their models. Three of them - feedback alignment, equilibrium propagation and predictive coding - have shown particular promise. Bengio and many others inspired by Hinton have been thinking about more biologically plausible learning mechanisms that might at least match the success of backpropagation.
0 kommentar(er)
