Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
78 72 83 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
64 43 83 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 0 promises resolved, 5 needed for a score.
All types combined: 77 (likely 71 to 82, high confidence), from 84 claims: 79 factual claims, 5 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Yann LeCun's claims
55 claims
“With probabilistic models, the training is confined to loss functions generated from the negative log-likelihood like the cross entropy.”
Anna Dawid and Yann LeCun, authors June 2023 Introduction to Latent Variable Energy-Based Models (Les Houches lecture notes) (arXiv)
With probabilistic models, training is confined to loss functions derived from the negative log-likelihood, such as cross entropy.
Mostly true True in substance, wrong in a detail.
What would count as true or false
This is a technical claim about machine learning, judged as of June 2023. True: the accepted understanding is that training a model as a probabilistic (normalized-likelihood) model means minimising negative log-likelihood or losses derived from it (cross-entropy, KL), with other losses falling outside the probabilistic framing. Mostly true: this is the standard view, with minor recognised exceptions (for example score matching or noise-contrastive estimation, which still serve as likelihood surrogates). Mixed: substantial and widely used probabilistic training methods do not derive from NLL, so the claim holds only for maximum-likelihood training. Mostly false: many standard probabilistic models are routinely trained with losses not derived from NLL (for example adversarial, score matching, proper scoring rules such as CRPS, Bayesian/variational objectives not reducible to NLL), so "confined" overstates a common practice. False: the evidence shows probabilistic models are not tied to NLL-type losses in general.
Evidence
- A Tutorial on Energy-Based Learning Courant Institute of Mathematical Sciences, NYU (to appear in Predicting Structured Data, MIT Press, 2006) · 19 August 2006 Own record
“Probabilistic learning is a special case of energy-based learning where the loss function is the negative log-likelihood, a.k.a. the maximum mutual information criterion.”
- A Tutorial on Energy-Based Learning Courant Institute of Mathematical Sciences, NYU (to appear in Predicting Structured Data, MIT Press, 2006) · 19 August 2006 Own record
“Estimating the parameters of a probabilistic model can be performed in a number of different ways, including maximum likelihood estimation with Bayes inversion, maximum conditional likelihood estimation, and (when possible) Bayesian averaging (possibly with variational approximations).”
- A Tutorial on Energy-Based Learning Courant Institute of Mathematical Sciences, NYU (to appear in Predicting Structured Data, MIT Press, 2006) · 19 August 2006 Own record
“Energy-based methods are equally well justified as probabilistic methods. They are merely designed for training models to answer a different kind of question than probabilistic models.”
- A Tutorial on Energy-Based Learning Courant Institute of Mathematical Sciences, New York University (authors LeCun, Chopra, Hadsell, Ranzato, Huang; chapter for MIT Press volume 'Predicting Structured Data') · 19 August 2006 Own record
“Estimating the parameters of a probabilistic model can be performed in a number of different ways, including maximum likelihood estimation with Bayes inversion, maximum conditional likelihood estimation, and (when possible) Bayesian averaging (possibly with variational approximations). Maximizing the conditional likelihood of the training samples is equivalent to minimizing what we called the negative log-likelihood loss.”
- A Tutorial on Energy-Based Learning Courant Institute of Mathematical Sciences, New York University (authors LeCun, Chopra, Hadsell, Ranzato, Huang; chapter for MIT Press volume 'Predicting Structured Data') · 19 August 2006 Own record
“Most probabilistic models can be viewed as special types of energy-based models in which the energy function satisfies certain normalizability conditions, and in which the loss function, optimized by learning, has a particular form.”
- A Tutorial on Energy-Based Learning Courant Institute of Mathematical Sciences, New York University (authors LeCun, Chopra, Hadsell, Ranzato, Huang; chapter for MIT Press volume 'Predicting Structured Data') · 19 August 2006 Own record
“Energy-based methods are equally well justified as probabilistic methods. They are merely designed for training models to answer a different kind of question than probabilistic models.”
- A Tutorial on Energy-Based Learning Courant Institute of Mathematical Sciences, New York University (authors LeCun, Chopra, Hadsell, Ranzato, Huang; chapter for MIT Press volume 'Predicting Structured Data') · 19 August 2006 Own record
“In this context, using an energy-based loss function other than the negative log-likelihood can be seen as a sampling method with a particular policy for picking the answers whose energy will be pulled up.”
- Estimation of Non-Normalized Statistical Models by Score Matching Journal of Machine Learning Research (JMLR), via jmlr.csail.mit.edu · 2005
“Here, we propose that such models can be estimated by minimizing the expected squared distance between the gradient of the log-density given by the model and the gradient of the log-density of the observed data.”
- Sliced Score Matching: A Scalable Approach to Density and Score Estimation Ermon Group (Stanford), ermongroup.github.io
“The de facto standard for learning a probability model is maximum likelihood (MLE). However, it is not suitable for learning unnormalized probability models, because the likelihood function”
- Sliced Score Matching: A Scalable Approach to Density and Score Estimation Ermon Group (Stanford), ermongroup.github.io
“Instead of using likelihood as the objective, score matching is based on the following objective”
Reasoning
An independent explainer describes maximum likelihood as the de facto standard for learning a probability model (E10), consistent with the claim that probabilistic training uses NLL-derived losses (E1, E5). However, score matching trains unnormalized probability models with a non-likelihood objective (E9, E11), and the subject's own tutorial lists Bayesian averaging, possibly with variational approximations, alongside maximum likelihood as ways to estimate probabilistic models (E3). These match the criteria's recognised minor exceptions, so "confined" slightly overstates the standard view.
Cite this claim
Yann LeCun, “With probabilistic models, training is confined to loss functions derived from the negative log-likelihood, such as cross entropy.” (said June 2023). Verdict: mostly true. Reliability API, claim c_bd7d985fd8d9, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_bd7d985fd8d9
Claims without a verdict
- Unresolvable 110
- Not yet due 3
- Not researched 309
- Not checkable 669
Is a verdict here wrong, or do you speak for Yann LeCun? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.