Deep learning uses the backpropagation algorithm to indicate how a machine should change the internal parameters that compute each layer's representation from the previous layer's.

Judge as of 2015-05-27. True: authoritative descriptions of deep learning (textbooks, review articles, standard training practice) show that deep multilayer networks are normally trained by computing gradients with backpropagation, and that these gradients determine how each layer's parameters are updated. Here 'parameters' means the parameters that map the previous layer's representation to the current one. Mostly true: backpropagation was the dominant method, but the claim's universal wording slightly overstates it because some deep learning methods of the time trained differently (for example, layerwise unsupervised pretraining without backpropagation). Mixed: backpropagation was used in only about half of mainstream deep learning practice. Mostly false: backpropagation was a minor or secondary method. False: deep learning did not, as a rule, use backpropagation to set layer parameters.

Set before the research began.

1. E1 Deep Neural Networks Springer (Apress open-access book chapter) · 27 April 2015

   “It is to be noted that the backpropagation algorithm is widely adopted.”

2. E2 Deep Neural Networks Springer (Apress open-access book chapter) · 27 April 2015

   “Because the update of the weights necessitates computing the gradient of the error (the cost function), it is essential for it to be differentiable.”

3. E3 Deep Neural Networks Springer (Apress open-access book chapter) · 27 April 2015

   “However, as the depth of the network increases, backpropagation’s performance degradation increases as well, making it unsuitable for training general deep architectures.”

4. E4 2010: Breakthrough of end-to-end deep learning. IDSIA (Jürgen Schmidhuber AI Blog) · 26 March 2021

   “However, in March 2010, our team with my outstanding Romanian postdoc Dan Ciresan [MLP1] showed that deep FNNs can indeed be trained end-to-end by plain backpropagation for important applications.”

A 2015 book chapter on deep neural networks says backpropagation is widely adopted (E1), and that weight updates require computing the gradient of the error (E2). This matches the mechanism in the claim. The same chapter also says backpropagation degrades as networks get deeper and is unsuitable for general deep architectures (E3). That suggests backpropagation was not the only way deep networks were trained, so the claim's universal wording may overstate the case somewhat. A later account says deep feedforward networks were shown in 2010 to be trainable end-to-end by plain backpropagation (E4).