ClaimsYann LeCun

“SEER outperformed the most advanced, state-of-the-art self-supervised systems, reaching 84.2 percent top-1 accuracy on ImageNet.”

Meta AI blog post, byline VP and Chief AI Scientist (LeCun) 4 March 2021 Self-supervised learning: The dark matter of intelligence (Meta AI)

Factual AI capabilities

After pretraining on a billion unlabeled Instagram images and fine-tuning on ImageNet, SEER outperformed state-of-the-art self-supervised systems, reaching 84.2 percent top-1 accuracy on ImageNet.

Resolved true The record shows it happened or was so.

Factual: Statements about the present or the past, checked against the record: did they hold up?

Said for an organisation they lead or work for; it counts on both records.

What would count as true or false

Judge as of 2021-03-04 against the SEER paper (Goyal et al., 2021, 'Self-supervised Pretraining of Visual Features in the Wild') and Meta/Facebook AI's announcement, applying any later published correction. True: the paper reports that SEER, pretrained on about a billion uncurated Instagram images and fine-tuned on ImageNet, reached 84.2% top-1 accuracy and beat the best prior self-supervised systems. Mostly true: as stated, but with a small discrepancy (e.g. accuracy within 0.2 points, or an image count described as roughly 1 billion). Mixed: the accuracy is right, but the claim that it beat state-of-the-art self-supervised systems does not hold, or vice versa. Mostly false: the 84.2% figure belongs to a different setup or model, or it was not the top result. False: the paper reports materially different results.

Set before the research began.

Evidence (3)

  1. E1 Self-supervised Pretraining of Visual Features in the Wild arXiv (Goyal et al., Meta AI research paper)

    “Our final SElf-supERvised (SEER) model, a RegNetY with 1.3B parameters trained on 1B random images with 512 GPUs achieves 84.2% top-1 accuracy, surpassing the best self-supervised pretrained model by 1% and confirming that self-supervised learning works in a real world setting.”
  2. E2 Self-supervised Pretraining of Visual Features in the Wild arXiv (Goyal et al., Meta AI research paper)

    “We achieve 84.2% top-1 accuracy on ImageNet, surpassing by +1%, the best existing pretrained model from SimCLRv2 [9].”
  3. E3 Facebook trains A.I. to 'see' using 1 billion public Instagram ... CNBC · 4 March 2021

    “It achieved a "classification accuracy score" of 84.2% when it attempted a test provided by ImageNet, which is a large visual database designed for use in visual object recognition software research.”

Reasoning

The SEER paper's abstract reports that SEER, a 1.3B-parameter RegNetY trained on 1B random images, reached 84.2% top-1 accuracy and beat the best self-supervised pretrained model by 1% (E1). In its comparison section, the paper names SimCLRv2 as the best prior pretrained model that SEER beat (E2). CNBC's report from the same date also gives the 84.2% ImageNet figure (E4). These match the claim on both the accuracy figure and the state-of-the-art comparison.

Cite this claim

Yann LeCun, “After pretraining on a billion unlabeled Instagram images and fine-tuning on ImageNet, SEER outperformed state-of-the-art self-supervised systems, reaching 84.2 percent top-1 accuracy on ImageNet.” (said 4 March 2021). Verdict: resolved true. Reliability API, claim c_b9b88e17b603, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_b9b88e17b603

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Yann LeCun, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More factual claims by Yann LeCun

  1. Mostly true Factual 18 March 2024
    “For starters, transformers are equivariant to permutations.”

    Transformers are equivariant to permutations, as ConvNets are equivariant to translations.

  2. Mostly true Factual 14 March 2024
    “If it were the case, we would have AI systems that could teach themselves to drive a car in 20 hours of practice, like any 17 year-old.”

    There are currently no AI systems that can teach themselves to drive a car in about 20 hours of practice, as a 17-year-old can.

  3. Mostly true Factual 8 March 2024
    “The architecture is good, the architecture of the encoder is good, but the fact that you train the system to reconstruct images does not lead it to produce to long, good generic features of images.”

    Training an encoder to reconstruct images does not lead it to produce good generic image features, though the encoder architecture is good.

  4. Mostly true Factual 8 March 2024
    “LLMs can do none of those or they can only do them in a very primitive way and they don’t really understand the physical world.”

    Current LLMs can do none of understanding the physical world, persistent memory, reasoning and planning, or only in a very primitive way.

  5. Mostly true Factual 8 March 2024
    “No, there’s one thing that autoregressive LLMs or that LLMs in general, not just the autoregressive one, but including the bird style bidirectional ones, are exploiting and its self supervised…”

    LLMs, including BERT-style bidirectional ones, exploit self-supervised learning, and LeCun has been a very strong advocate of self-supervised learning for many years.

All of them

Verdicts are made automatically; how verdicts are made.