Gary Marcus

Factual · AI capabilities

“Deep learning doesn’t just do, let’s say, linear regression, which is a form of correlation.”

Gary Marcus, interviewee · 20 September 2019 · Gary Marcus on Why AI Needs a Reboot (Psychology Today)

Deep learning does more than linear regression but is a more sophisticated form of statistical correlation.

Mostly true True in substance, wrong in a detail.

The second judge would not resolve this claim; the verdict shown is the first judge's.

What would count as true or false

Judge as of 2019-09-20 against the machine-learning literature. The claim is that deep learning goes beyond linear regression but is still a more sophisticated form of statistical correlation or pattern-fitting. True: standard technical descriptions treat deep learning as nonlinear statistical function approximation fitted to correlations in training data, distinct from and more powerful than linear regression. Mostly true: the characterisation is broadly accepted, though 'correlation' is somewhat loose, since the methods fit nonlinear functions. Mixed: experts are substantially divided on whether 'correlation' adequately describes it. Mostly false: deep learning does exceed linear regression, but the evidence shows calling it correlation materially mischaracterises it. False: the evidence shows deep learning is not a statistical, data-fitting method.

Evidence

1. Goodfellow et al.: Deep Learning Book Chapter 6 deeplearningbook.org (Goodfellow, Bengio, Courville, textbook chapter)

   “Clearly, we must use a nonlinear function to describe the features.”

2. Goodfellow et al.: Deep Learning Book Chapter 6 deeplearningbook.org (Goodfellow, Bengio, Courville, textbook chapter)

   “Linear models also have the obvious defect that the model capacity is limited to linear functions, so the model cannot understand the interaction between any two input variables.”

3. Deep learning - Nature Nature · 27 May 2015

   “Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction.”

4. A Survey on Universal Approximation Theorems arXiv (ar5iv rendering)

   “From a mathematical point of view, neurons are made of compositions of a nonlinear function (also called activation function) and a linear function [4].”

Reasoning

The Goodfellow et al. textbook (E1, E2) says linear models can only represent linear functions, and that deep feedforward networks use nonlinear functions to get past this limit. That supports the claim that deep learning goes beyond linear regression. E3 and E4 describe deep learning as layered compositions of linear and nonlinear functions that learn representations from data, which fits a description of it as statistical function approximation fitted to data. None of the sources uses the word "correlation", though, and they stress nonlinear representation learning, so the "more sophisticated correlation" wording is broadly consistent with the evidence but loose. That matches the criteria's definition of mostly true.

Cite this claim

Gary Marcus, “Deep learning does more than linear regression but is a more sophisticated form of statistical correlation.” (said 20 September 2019). Verdict: mostly true. Reliability API, claim c_8d0f6de14388, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_8d0f6de14388