Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
83 77 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
67 49 83 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 1 promise resolved, 5 needed for a score.
All types combined: 81 (likely 76 to 86, high confidence), from 87 claims: 79 factual claims, 8 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Rodney Brooks's claims
42 claims
“Typically one would show it millions of images (yes, millions), for which there was ground truth known about which images contained what objects.”
Rodney Brooks, roboticist, writing on his blog 15 July 2018 [FoR&AI] Steps Toward Super Intelligence I, How We Got Here (rodneybrooks.com)
Training an image-recognition neural network typically involves showing it millions of labelled images with known ground truth.
Resolved true The record shows it happened or was so.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judged as of 2018-07-15, as a statement about typical practice. True: the technical literature shows that image-recognition deep networks of that era (e.g. those trained on ImageNet, about 1.2 million labelled images in ILSVRC) were typically trained on on the order of millions of labelled images with known ground truth. Mostly true: typical training sets were large and labelled but often in the hundreds of thousands, or relied on pretraining on million-image sets, so 'millions' is a slight overstatement. Mixed: practice varied widely, with roughly as many typical cases using far fewer images. Mostly false: million-image labelled training was the exception, not the typical case. False: image-recognition networks were not typically trained on labelled data at that scale.
Evidence
- ImageNet Classification with Deep Convolutional Neural Networks NeurIPS Proceedings (Krizhevsky, Sutskever, Hinton, University of Toronto) · 2012
“We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes.”
- ImageNet Classification with Deep Convolutional Neural Networks NeurIPS Proceedings (Krizhevsky, Sutskever, Hinton, University of Toronto) · 2012
“Until recently, datasets of labeled images were relatively small — on the order of tens of thousands of images”
- ImageNet Large Scale Visual Recognition Challenge arXiv (Russakovsky et al.; ar5iv rendering)
“There are 1000 object classes and approximately 1.2 million training images, 50 thousand validation images and 100 thousand test images.”
- ImageNet Large Scale Visual Recognition Challenge arXiv (Russakovsky et al.; ar5iv rendering)
“Caltech 101 (Fei-Fei et al., 2004) was among the first standardized datasets for multi-category image classification, with 101 object classes and commonly 15-30 training images per class.”
Reasoning
The 2012 Krizhevsky, Sutskever and Hinton paper trained a deep convolutional network on the 1.2 million labelled images of the ImageNet LSVRC-2010 contest, sorted into 1000 classes (E1). The ILSVRC benchmark paper confirms about 1.2 million labelled training images across 1000 classes (E3). This matches the resolution criteria's own example of training "on the order of millions" of labelled images with known ground truth. E2 and E4 show that labelled datasets were much smaller before ImageNet, but that was earlier practice, not the deep-network practice of the period the claim describes.
Cite this claim
Rodney Brooks, “Training an image-recognition neural network typically involves showing it millions of labelled images with known ground truth.” (said 15 July 2018). Verdict: resolved true. Reliability API, claim c_343bea0e2fef, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_343bea0e2fef
Claims without a verdict
Is a verdict here wrong, or do you speak for Rodney Brooks? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.