After pretraining on a billion unlabeled Instagram images and fine-tuning on ImageNet, SEER outperformed state-of-the-art self-supervised systems, reaching 84.2 percent top-1 accuracy on ImageNet.
Resolved true The record shows it happened or was so.
Factual: Statements about the present or the past, checked against the record: did they hold up?
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judge as of 2021-03-04 against the SEER paper (Goyal et al., 2021, 'Self-supervised Pretraining of Visual Features in the Wild') and Meta/Facebook AI's announcement, applying any later published correction. True: the paper reports that SEER, pretrained on about a billion uncurated Instagram images and fine-tuned on ImageNet, reached 84.2% top-1 accuracy and beat the best prior self-supervised systems. Mostly true: as stated, but with a small discrepancy (e.g. accuracy within 0.2 points, or an image count described as roughly 1 billion). Mixed: the accuracy is right, but the claim that it beat state-of-the-art self-supervised systems does not hold, or vice versa. Mostly false: the 84.2% figure belongs to a different setup or model, or it was not the top result. False: the paper reports materially different results.
Set before the research began.
Evidence (3)
-
E1 Self-supervised Pretraining of Visual Features in the Wild arXiv (Goyal et al., Meta AI research paper)
“Our final SElf-supERvised (SEER) model, a RegNetY with 1.3B parameters trained on 1B random images with 512 GPUs achieves 84.2% top-1 accuracy, surpassing the best self-supervised pretrained model by 1% and confirming that self-supervised learning works in a real world setting.”
-
E2 Self-supervised Pretraining of Visual Features in the Wild arXiv (Goyal et al., Meta AI research paper)
“We achieve 84.2% top-1 accuracy on ImageNet, surpassing by +1%, the best existing pretrained model from SimCLRv2 [9].”
-
E3 Facebook trains A.I. to 'see' using 1 billion public Instagram ... CNBC · 4 March 2021
“It achieved a "classification accuracy score" of 84.2% when it attempted a test provided by ImageNet, which is a large visual database designed for use in visual object recognition software research.”
Reasoning
The SEER paper's abstract reports that SEER, a 1.3B-parameter RegNetY trained on 1B random images, reached 84.2% top-1 accuracy and beat the best self-supervised pretrained model by 1% (E1). In its comparison section, the paper names SimCLRv2 as the best prior pretrained model that SEER beat (E2). CNBC's report from the same date also gives the 84.2% ImageNet figure (E4). These match the claim on both the accuracy figure and the state-of-the-art comparison.
Cite this claim
Yann LeCun, “After pretraining on a billion unlabeled Instagram images and fine-tuning on ImageNet, SEER outperformed state-of-the-art self-supervised systems, reaching 84.2 percent top-1 accuracy on ImageNet.” (said 4 March 2021). Verdict: resolved true. Reliability API, claim c_b9b88e17b603, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_b9b88e17b603
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Yann LeCun
-
Mostly true
“For starters, transformers are equivariant to permutations.”
Transformers are equivariant to permutations, as ConvNets are equivariant to translations.
-
Mostly true
“If it were the case, we would have AI systems that could teach themselves to drive a car in 20 hours of practice, like any 17 year-old.”
-
Mostly true
“The architecture is good, the architecture of the encoder is good, but the fact that you train the system to reconstruct images does not lead it to produce to long, good generic features of images.”
-
Mostly true
“LLMs can do none of those or they can only do them in a very primitive way and they don’t really understand the physical world.”
-
Mostly true
“No, there’s one thing that autoregressive LLMs or that LLMs in general, not just the autoregressive one, but including the bird style bidirectional ones, are exploiting and its self supervised…”
Verdicts are made automatically; how verdicts are made.