Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
78 72 83 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
64 43 83 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 0 promises resolved, 5 needed for a score.
All types combined: 77 (likely 71 to 82, high confidence), from 84 claims: 79 factual claims, 5 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Yann LeCun's claims
2 claims
| Verdict | Kind | Said | Claim | Topic |
|---|---|---|---|---|
| Mostly false | Factual | 2019 | Users upload 2 to 3 billion photos on Facebook every day. Factual · 2019 | AI compute, hardware and data |
| Mostly false | Factual | 15 May 2014 | Convolutional nets were always the record holder on MNIST, and SVMs never really managed to beat them there. Factual · 15 May 2014 | AI capabilities |
“It’s important to remind people that convolutional nets were always the record holder on MNIST.”
Yann LeCun, in Reddit AMA 15 May 2014 Yann LeCun's answers from the Reddit AMA (FastML)
Convolutional nets were always the record holder on MNIST, and SVMs never really managed to beat them there.
Mostly false False in substance, right in a detail.
What would count as true or false
Judge as of 2014-05-15. The claim says that at every point in MNIST's history up to then, the best reported test error was held by a convolutional network, and that an SVM never held it. Compare dated best-reported results from the MNIST benchmark history: LeCun's MNIST page and the original papers. True: at every date from the benchmark's introduction to May 2014, the lowest published error belonged to a convolutional-net method. Variants that combine convnets with distortions, ensembles or boosting still count as convnets. No SVM or other non-convolutional method ever had a lower published error than the best convnet result published by that date. Mostly true: convnets held the record throughout, except that a non-convnet method briefly matched it (a tie) or led only by a negligible amount within the same reporting precision. No SVM led outright. Mixed: the answer depends on the category. Convnets held the overall record throughout, but an SVM or other method held the best result in a commonly reported sub-category (for example, no data augmentation, or no prior knowledge) for a substantial period. Mostly false: an SVM (for example a virtual or deskewed SVM) or another non-convnet method held the overall lowest published error for some period, even though convnets held it most of the time. False: non-convnet methods held the overall record for long stretches, or convnets were not the record holder for most of the period.
Evidence
- Training Invariant Support Vector Machines Machine Learning (Springer / Kluwer), 46, 161–190 · 2002
“One of the significant new results reported in this work is our recent achievement of the lowest reported test error on the well-known MNIST digit recognition benchmark task, with SVM training”
- Training Invariant Support Vector Machines Machine Learning (Springer / Kluwer), 46, 161–190 · 2002
“Prior to the present work, the best system on the MNIST set was a boosted ensemble of LeNet4 neural networks, trained on a huge database of artificially generated virtual examples.”
- MNIST database - Wikipedia Wikipedia · 17 August 2013
“Sometimes, these systems can be very successful; one such system achieved an error rate on the database of 0.39 percent.”
- HeroSvm 2.1 | Centre for Pattern Recognition and Machine Intelligence - Concordia University CENPARMI, Concordia University · 4 August 2005
“| VSVMb [5][21] | 0.38 |”
Reasoning
The 2002 peer-reviewed paper on invariant SVMs [E1] reports that its SVM training reached the lowest reported test error on MNIST. According to the same paper [E2], the previous best system was a boosted ensemble of LeNet4 neural networks. The paper reports that its SVM achieved the lowest reported MNIST test error at the time [E1], which, if accurate, goes against the claim that convnets were always the record holder. The evidence does not show that SVMs led for long stretches, and E3 and E4 do not clearly name methods, dates or what their figures measure. That fits the 'mostly false' category better than 'false'.
Cite this claim
Yann LeCun, “Convolutional nets were always the record holder on MNIST, and SVMs never really managed to beat them there.” (said 15 May 2014). Verdict: mostly false. Reliability API, claim c_6199a2b9c2a8, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_6199a2b9c2a8
Claims without a verdict
- Unresolvable 110
- Not yet due 3
- Not researched 309
- Not checkable 669
Is a verdict here wrong, or do you speak for Yann LeCun? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.