Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
78 72 83 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
64 43 83 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 0 promises resolved, 5 needed for a score.
All types combined: 77 (likely 71 to 82, high confidence), from 84 claims: 79 factual claims, 5 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Yann LeCun's claims
55 claims
“No, there’s one thing that autoregressive LLMs or that LLMs in general, not just the autoregressive one, but including the bird style bidirectional ones, are exploiting and its self supervised running, and I’ve been a very, very strong advocate of self supervised running for many years.”
Yann LeCun, Meta chief AI scientist, podcast guest 8 March 2024 Transcript for Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI (Lex Fridman Podcast)
LLMs, including BERT-style bidirectional ones, exploit self-supervised learning, and LeCun has been a very strong advocate of self-supervised learning for many years.
Mostly true True in substance, wrong in a detail.
The second judge said resolved true; the verdict shown is the first judge's.
What would count as true or false
The claim has two parts: (a) LLMs, both autoregressive and BERT-style bidirectional, are trained with self-supervised learning (e.g., next-token prediction or masked-token prediction on unlabeled text); (b) LeCun had been a very strong public advocate of self-supervised learning for many years before March 2024 (several years at least, e.g., from the mid-2010s or earlier). TRUE: both parts hold. MOSTLY TRUE: both hold in substance with a minor slip (e.g., his advocacy is documented for somewhat fewer years than the wording suggests, but still several years). MIXED: one part holds and the other clearly does not. MOSTLY FALSE: (a) holds only for some model types, and (b) is weak or recent. FALSE: standard LLM pretraining is shown not to be self-supervised, and LeCun is not shown to have advocated it before.
Evidence
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding arXiv (ar5iv rendering) · 3 April 2018
“BERT is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers.”
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding arXiv (ar5iv rendering) · 3 April 2018
“The training loss is the sum of the mean masked LM likelihood and the mean next sentence prediction likelihood.”
- Improving Language Understanding by Generative Pre-Training OpenAI (Radford, Narasimhan, Salimans, Sutskever)
“First, we use a language modeling objective on the unlabeled data to learn the initial parameters of a neural network model.”
- AI chip evolution will soon take new direction says AI pioneer - Embedded Embedded.com (sister site of EE Times) · 19 February 2019
““In my opinion, the future of AI is self-supervised learning,” said Yann LeCun, who is considered the father of convolutional neural nets (CNNs) now used widely in computer vision and other systems.”
Reasoning
The BERT paper says BERT is pre-trained on unlabeled text using a masked language modelling objective (E1, E2). The original GPT paper describes a language modelling objective on unlabeled data (E3). Together these support part (a) for both bidirectional and autoregressive models. For part (b), a 2019 Embedded.com article quotes LeCun saying "the future of AI is self-supervised learning" (E4), which shows public advocacy about five years before March 2024. That is several years, but the evidence does not show advocacy going back to the mid-2010s or earlier, so the verdict is mostly true rather than true.
Cite this claim
Yann LeCun, “LLMs, including BERT-style bidirectional ones, exploit self-supervised learning, and LeCun has been a very strong advocate of self-supervised learning for many years.” (said 8 March 2024). Verdict: mostly true. Reliability API, claim c_c1d6510692db, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_c1d6510692db
Claims without a verdict
- Unresolvable 110
- Not yet due 3
- Not researched 309
- Not checkable 669
Is a verdict here wrong, or do you speak for Yann LeCun? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.