Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
78 72 83 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
64 43 83 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 0 promises resolved, 5 needed for a score.
All types combined: 77 (likely 71 to 82, high confidence), from 84 claims: 79 factual claims, 5 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Yann LeCun's claims
23 claims
“SEER outperformed the most advanced, state-of-the-art self-supervised systems, reaching 84.2 percent top-1 accuracy on ImageNet.”
Meta AI blog post, byline VP and Chief AI Scientist (LeCun) 4 March 2021 Self-supervised learning: The dark matter of intelligence (Meta AI)
After pretraining on a billion unlabeled Instagram images and fine-tuning on ImageNet, SEER outperformed state-of-the-art self-supervised systems, reaching 84.2 percent top-1 accuracy on ImageNet.
Resolved true The record shows it happened or was so.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judge as of 2021-03-04 against the SEER paper (Goyal et al., 2021, 'Self-supervised Pretraining of Visual Features in the Wild') and Meta/Facebook AI's announcement, applying any later published correction. True: the paper reports that SEER, pretrained on about a billion uncurated Instagram images and fine-tuned on ImageNet, reached 84.2% top-1 accuracy and beat the best prior self-supervised systems. Mostly true: as stated, but with a small discrepancy (e.g. accuracy within 0.2 points, or an image count described as roughly 1 billion). Mixed: the accuracy is right, but the claim that it beat state-of-the-art self-supervised systems does not hold, or vice versa. Mostly false: the 84.2% figure belongs to a different setup or model, or it was not the top result. False: the paper reports materially different results.
Evidence
- Self-supervised Pretraining of Visual Features in the Wild arXiv (Goyal et al., Meta AI research paper)
“Our final SElf-supERvised (SEER) model, a RegNetY with 1.3B parameters trained on 1B random images with 512 GPUs achieves 84.2% top-1 accuracy, surpassing the best self-supervised pretrained model by 1% and confirming that self-supervised learning works in a real world setting.”
- Self-supervised Pretraining of Visual Features in the Wild arXiv (Goyal et al., Meta AI research paper)
“We achieve 84.2% top-1 accuracy on ImageNet, surpassing by +1%, the best existing pretrained model from SimCLRv2 [9].”
- Facebook trains A.I. to 'see' using 1 billion public Instagram ... CNBC · 4 March 2021
“It achieved a "classification accuracy score" of 84.2% when it attempted a test provided by ImageNet, which is a large visual database designed for use in visual object recognition software research.”
Reasoning
The SEER paper's abstract reports that SEER, a 1.3B-parameter RegNetY trained on 1B random images, reached 84.2% top-1 accuracy and beat the best self-supervised pretrained model by 1% (E1). In its comparison section, the paper names SimCLRv2 as the best prior pretrained model that SEER beat (E2). CNBC's report from the same date also gives the 84.2% ImageNet figure (E4). These match the claim on both the accuracy figure and the state-of-the-art comparison.
Cite this claim
Yann LeCun, “After pretraining on a billion unlabeled Instagram images and fine-tuning on ImageNet, SEER outperformed state-of-the-art self-supervised systems, reaching 84.2 percent top-1 accuracy on ImageNet.” (said 4 March 2021). Verdict: resolved true. Reliability API, claim c_b9b88e17b603, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_b9b88e17b603
Claims without a verdict
- Unresolvable 110
- Not yet due 3
- Not researched 309
- Not checkable 669
Is a verdict here wrong, or do you speak for Yann LeCun? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.