Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
78 72 83 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
64 43 83 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 0 promises resolved, 5 needed for a score.
All types combined: 77 (likely 71 to 82, high confidence), from 84 claims: 79 factual claims, 5 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Yann LeCun's claims
55 claims
“We are starting to see impressive results in natural language processing with Deep Learning augmented with a memory module.”
Yann LeCun, Facebook AI director, interviewed 18 February 2015 Facebook AI Director Yann LeCun on His Quest to Unleash Deep Learning (IEEE Spectrum)
Deep Learning systems augmented with a memory module are starting to show impressive results in natural language processing.
Mostly true True in substance, wrong in a detail.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judge as of February 2015. TRUE: by 18 February 2015, work had been published or posted showing deep networks augmented with an explicit memory module (for example, Memory Networks or Neural Turing Machines) achieving notable results on NLP tasks such as question answering, and these results were regarded as impressive and emerging. MOSTLY TRUE: results existed, but mainly on synthetic or toy NLP tasks rather than on real-world benchmarks. MOSTLY FALSE: memory-augmented models existed, but had shown only marginal or non-NLP results. FALSE: no memory-augmented deep learning NLP results had appeared by that date.
Evidence
- Memory Networks arXiv (rendered via ar5iv.labs.arxiv.org)
“We investigate these models in the context of question answering (QA) where the long-term memory effectively acts as a (dynamic) knowledge base, and the output is a textual response. We evaluate them on a large-scale QA task, and a smaller, but more complex, toy task generated from a simulated world.”
- Memory Networks arXiv (rendered via ar5iv.labs.arxiv.org)
“Some answers appear fine, whereas others are nonsensical.”
- Memory Networks arXiv (rendered via ar5iv.labs.arxiv.org)
“Our work was submitted to arxiv just before the Neural Turing Machine work of Graves et al. 2014, which is one of the most relevant related methods.”
- Neural Turing Machines arXiv (rendered via ar5iv.labs.arxiv.org)
“Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples.”
Reasoning
E3 shows that the Memory Networks preprint was submitted to arXiv in 2014, just before the Graves et al. 2014 Neural Turing Machine work, which is before the claim date. It tested a memory-augmented network on question answering, using a large-scale QA task and a toy task generated from a simulated world. The evidence does not show these results being described as impressive: the paper itself (E2) says some answers appear fine while others are nonsensical. The Neural Turing Machine paper (E4) reports preliminary results on algorithmic tasks such as copying, sorting and associative recall; the quoted passage does not mention language results. Memory-augmented NLP results clearly existed by February 2015, but nothing here shows they were seen as impressive on real-world benchmarks, so the claim holds largely but not fully.
Cite this claim
Yann LeCun, “Deep Learning systems augmented with a memory module are starting to show impressive results in natural language processing.” (said 18 February 2015). Verdict: mostly true. Reliability API, claim c_24d8dbb268c8, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_24d8dbb268c8
Claims without a verdict
- Unresolvable 110
- Not yet due 3
- Not researched 309
- Not checkable 669
Is a verdict here wrong, or do you speak for Yann LeCun? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.