Claims

Prediction · AI capabilities

“one of the major challenges of computer vision, machine learning, and AI in general in the next decade will be to devise methods that can automatically learn good features hierarchies from unlabeled and labeled data in an integrated fashion.”

Yann LeCun, co-author of paper · 2010 · Convolutional Networks and Applications in Vision (yann.lecun.com)

In the next decade, a major challenge of computer vision, machine learning and AI will be devising methods that learn good feature hierarchies from unlabeled and labeled data in an integrated fashion.

Mostly true True in substance, wrong in a detail.

The second judge would not resolve this claim; the verdict shown is the first judge's.

What would count as true or false

Judge over 2010–2020. True: during that decade, learning good hierarchical features or representations from unlabeled and labeled data together was a widely recognised major research challenge in computer vision, machine learning or AI. Signs include a large research effort on unsupervised, self-supervised and semi-supervised representation learning, prominent venues such as ICLR, and surveys or leading researchers naming it a central open problem. Mostly true: it was a recognised major challenge, but mainly in a form slightly different from 'integrated' labeled-plus-unlabeled learning (for example pure self-supervised pretraining followed by fine-tuning). Mixed: it was an important research topic, but it was widely seen as secondary to purely supervised learning for much of the decade. Mostly false: the field largely set the problem aside as a minor concern. False: contemporary evidence shows the field regarded it as solved or unimportant throughout the decade.

Evidence

1. Representation Learning: A Review and New Perspectives ar5iv (arXiv mirror)

   “Future work is hopefully going to help identify which of these elements matter most, how to generalize them across a large variety of tasks and architectures, and in particular contexts where most examples are unlabeled, i.e., including an unsupervised component in the training criterion.”

2. Representation Learning: A Review and New Perspectives ar5iv (arXiv mirror)

   “Researchers report than in such conditions, layerwise unsupervised pre-training brought little or no improvement over pure supervised learning from scratch when training for long enough.”

3. Semi-Supervised and Unsupervised Deep Visual Learning: A Survey ar5iv (arXiv mirror)

   “Built upon different assumptions, these paradigms are often developed independently, whilst sharing the same aim to learn more powerful representations and models using unlabeled data.”

4. Semi-Supervised and Unsupervised Deep Visual Learning: A Survey ar5iv (arXiv mirror)

   “SimCLR [12] and SimCLRv2 [72] are follow-up works introducing self-supervised contrastive learning for task-agnostic unsupervised pre-training, followed by supervised or semi-supervised fine-tuning with label supervision as the downstream task.”

Reasoning

The representation-learning review (E1) names learning in contexts where most examples are unlabeled, with an unsupervised component in training, as key future work. A later survey (E3) describes several semi-supervised and unsupervised paradigms, all aimed at learning better representations from unlabeled data, which indicates a large research effort. Some of that effort, such as SimCLR and SimCLRv2 (E4), took the form of self-supervised pre-training followed by supervised or semi-supervised fine-tuning, rather than integrated labeled-plus-unlabeled learning. E2 notes that in certain conditions, and when training for long enough, layerwise unsupervised pre-training brought little or no improvement over supervised training from scratch, but the overall evidence best fits the mostly-true criterion.

Cite this claim

Yann LeCun, “In the next decade, a major challenge of computer vision, machine learning and AI will be devising methods that learn good feature hierarchies from unlabeled and labeled data in an integrated fashion.” (said 2010). Verdict: mostly true. Reliability API, claim c_2d4d6f7aabf3, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_2d4d6f7aabf3