Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
83 77 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
67 49 83 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 1 promise resolved, 5 needed for a score.
All types combined: 81 (likely 76 to 86, high confidence), from 87 claims: 79 factual claims, 8 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Rodney Brooks's claims
42 claims
“Oh, this is already happening”
Rodney Brooks, blogger 1 January 2018 My Dated Predictions (rodneybrooks.com)
Academic rumblings about the limits of Deep Learning are already happening as of 2018.
Resolved true The record shows it happened or was so.
What would count as true or false
Judge as of 2018-01-01. True: by the end of 2017 or start of 2018, published academic work or prominent academic commentary criticised or pointed to limits of deep learning (for example Gary Marcus's 'Deep Learning: A Critical Appraisal', posted January 2018, or earlier papers and talks by researchers on its limits). Mostly true: such criticism existed but was mainly from a few individuals or appeared just around the claim date. Mixed: critiques existed only outside academia (press, industry), with little academic content. Mostly false: only isolated or tangential remarks existed, overstated as a trend. False: there is evidence that no such academic critique of deep learning's limits existed by early 2018.
Evidence
- Deep Learning: A Critical Appraisal arXiv (author: Gary Marcus, New York University)
“Yet deep learning may well be approaching a wall, much as I anticipated earlier, at beginning of the resurgence (Marcus, 2012), and as leading figures like Hinton (Sabour, Frosst, & Hinton, 2017) and Chollet (2017) have begun to imply in recent months.”
- Failures of Gradient-Based Deep Learning PMLR (Proceedings of Machine Learning Research), ICML 2017 · 17 July 2017
“We describe four types of simple problems, for which the gradient-based algorithms commonly used in deep learning either fail or suffer from significant difficulties.”
- An Interview with François Chollet DataCamp · 18 December 2017
“One important limitation of our capabilities on that front is that our models can only handle inputs that are extremely close to what they've seen before -- you can't get very far from your training data.”
- Predictions Scorecard, 2019 January 01 rodneybrooks.com (Rodney Brooks) · 1 January 2019 Own record
“There were plenty of papers published on limits of Deep Learning.”
Reasoning
E2 is a peer-reviewed ICML 2017 paper, published July 2017, that describes simple problems on which the gradient-based methods common in deep learning "either fail or suffer from significant difficulties". That is academic work on the limits of deep learning from before the claim date. E1 is Gary Marcus's critical appraisal, the example named in the resolution criteria, and it cites recent remarks on deep learning's limits by Hinton (Sabour, Frosst & Hinton, 2017) and Chollet (2017). In E3, an interview from December 2017, Chollet says models cannot get far from their training data. Taken together, the independent sources show academic criticism of deep learning's limits existed by the start of 2018, so the claim resolves true.
Cite this claim
Rodney Brooks, “Academic rumblings about the limits of Deep Learning are already happening as of 2018.” (said 1 January 2018). Verdict: resolved true. Reliability API, claim c_125b965f7967, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_125b965f7967
Claims without a verdict
Is a verdict here wrong, or do you speak for Rodney Brooks? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.