Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
78 72 83 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
64 43 83 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 0 promises resolved, 5 needed for a score.
All types combined: 77 (likely 71 to 82, high confidence), from 84 claims: 79 factual claims, 5 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Yann LeCun's claims
23 claims
“However, each iteration of negative log-likelihood loss involves considerably more work because pushing up every incorrect answer is computationally expensive when no analytical expression for the derivative of the second term exists.”
LeCun et al., tutorial authors 2006 A Tutorial on Energy-Based Learning (yann.lecun.com)
This quote could not be matched word for word to the source page.
Each NLL iteration involves considerably more work because pushing up every incorrect answer is expensive when the second term's derivative has no analytical form.
Resolved true The record shows it happened or was so.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judged as of 2006, as a technical claim about energy-based learning. True: the standard analysis (for example, LeCun et al. 2006, 'A Tutorial on Energy-Based Learning') shows that the gradient of the NLL loss has a second term, an expectation over all possible answers weighted by their Gibbs probabilities, which in general has no closed form and needs costly computation or approximation (such as sampling or summing over the answer space), so each NLL iteration costs considerably more than losses that push up only one or a few contrastive answers. Mostly true: the reasoning is correct, but with a minor imprecision, for example the cost depends on the model, and some structured models allow tractable computation. Mixed: the extra cost holds only in some settings that make up roughly half of the cases discussed. Mostly false: the extra cost is real only in narrow cases, and the general claim overstates it. False: NLL iterations are not generally more expensive for this reason.
Evidence
- 18 Confronting the Partition Function deeplearningbook.org (Goodfellow, Bengio and Courville, Deep Learning) · 1 January 2012
“This operation is intractable for many interesting models.”
- 18 Confronting the Partition Function deeplearningbook.org (Goodfellow, Bengio and Courville, Deep Learning) · 1 January 2012
“Each of these approximations can be understood as making the negative phase computationally cheaper but also making it push down in the wrong locations.”
- To Relieve Your Headache of Training an MRF, Take AdVIL ICLR 2020 (Li, Du, Xu, Welling, Zhu, Zhang) · 2020
“Unfortunately, both the NLL in Eqn. (1) and its gradient in Eqn. (2) are intractable in general for two reasons.”
Reasoning
The Deep Learning textbook's chapter on the partition function says the operation it discusses is intractable for many interesting models (E1), and that approximations make the negative phase computationally cheaper (E2); an ICLR 2020 paper states that the NLL and its gradient are intractable in general (E3). They also say that the usual approximations exist to make the negative phase cheaper to compute (E2). The ICLR 2020 paper says the NLL and its gradient are intractable in general (E3). The claim applies only when the second term has no analytical form, so it already allows for the tractable cases, and the evidence supports the standard analysis that exact NLL updates cost considerably more.
Cite this claim
Yann LeCun, “Each NLL iteration involves considerably more work because pushing up every incorrect answer is expensive when the second term's derivative has no analytical form.” (said 2006). Verdict: resolved true. Reliability API, claim c_c68de5ffea08, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_c68de5ffea08
Claims without a verdict
- Unresolvable 110
- Not yet due 3
- Not researched 309
- Not checkable 669
Is a verdict here wrong, or do you speak for Yann LeCun? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.