| Not researched |
Factual |
30 Mar 2024 |
Anti-vaccine propaganda causes people to die needlessly.
Factual · 30 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Factual |
26 Mar 2024 |
Crypto money is secretly funding AI doomerism and lobbying for AI regulations, effectively working against open source AI platforms.
Factual · 26 Mar 2024 |
AI policy and governance |
| Not researched |
Factual |
20 Mar 2024 |
The CodeGen team in FAIR-Paris at Meta is recruiting.
Factual · 20 Mar 2024 |
Field building and careers |
| Not researched |
Factual |
14 Mar 2024 |
As of March 2024 there is no fully autonomous, reliable self-driving, even though millions of hours of labeled training data exist.
Factual · 14 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
14 Mar 2024 |
The report of the Committee on AI appointed by the French government is available for download, with a summary in English.
Factual · 14 Mar 2024 |
AI policy and governance |
| Not researched |
Factual |
14 Mar 2024 |
Yann LeCun is a member of the Committee on AI appointed by the French government.
Factual · 14 Mar 2024 |
AI policy and governance |
| Not researched |
Factual |
13 Mar 2024 |
The French AI Commission's report mentions the importance of open models.
Factual · 13 Mar 2024 |
AI policy and governance |
| Not researched |
Factual |
13 Mar 2024 |
The French Commission on Artificial Intelligence delivered a report to the President of the Republic containing 25 recommendations, 7 of them priority recommendations.
Factual · 13 Mar 2024 |
AI policy and governance |
| Not researched |
Factual |
10 Mar 2024 |
The adaptive equalizer is basically a linear classifier trained with least squares, a direct descendant of Bernie Widrow's Adaline, a competitor of the Perceptron.
Factual · 10 Mar 2024 |
Technology |
| Not researched |
Factual |
8 Mar 2024 |
Wav2Vec-based systems can produce multilingual speech recognition using mostly unlabeled data and only a few minutes of labeled data.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Humans spend more time solving complex problems or questions because they are more difficult.
Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Factual |
8 Mar 2024 |
Throughout history, technological revolutions and cultural phenomena were accompanied by media reactions blaming society's problems on the change.
Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Factual |
8 Mar 2024 |
Asked how this is done for visual data, LeCun said that he and his colleagues have been doing it with I-JEPA architectures.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
RLHF reward models are used only for training LLMs, not for inference.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Human feedback is what had the transformational effect on large language models, and it can be used in many ways, some purely supervised.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Meta published its V-JEPA work the week before this interview, as a first step toward training AI systems on video.
Factual · 8 Mar 2024 |
Their own work or organisation |
| Not researched |
Factual |
8 Mar 2024 |
After abandoning pixel prediction, joint embedding with prediction in representation space worked for learning representations.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
The success of the US tech industry and Silicon Valley is due to information circulating and disseminating very quickly.
Factual · 8 Mar 2024 |
Economics and markets |
| Not researched |
Factual |
8 Mar 2024 |
The Catholic Church disliked the printing press but had no choice but to accept it.
Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Factual |
8 Mar 2024 |
Most of what humans learn and know comes from observation and interaction with the real world, not from language.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Attempts to train systems to predict video pixels (including latent-variable approaches) have been a complete failure essentially.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
LLMs do not plan their answers before producing them; they produce one word after another.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
There are now about six different non-contrastive methods for preventing collapse in joint embedding training.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
I-JEPA does not need to know the input is an image, since it only uses masking, whereas DINO relies on image-specific transformations.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
In the first few months of life, babies have little influence on the world and can only observe.
Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Factual |
8 Mar 2024 |
The printing press transformed the world by making people smarter and better informed.
Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Factual |
8 Mar 2024 |
One set of non-contrastive techniques is based on distillation, including DeepMind's BYOL and FAIR's I-JEPA.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Researchers currently have no idea how to do learned hierarchical planning in AI.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
LLMs with vision extensions are not trained on video.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
A joint embedding system trained only to match representations of an image and its corrupted version collapses, ignoring the input and producing constant representations.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Gradient-based optimization can only be done in continuous spaces with differentiable functions.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Actually building a chemical weapon or bioweapon is much harder than obtaining instructions, and an LLM will not help with that step.
Factual · 8 Mar 2024 |
Biosecurity |
| Not researched |
Factual |
8 Mar 2024 |
For about 10 years, LeCun's team tried to learn image and video representations with generative models that predict pixels, and failed.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
No one in the world, including countries, has used bioweapons, and they are banned by international treaties.
Factual · 8 Mar 2024 |
Biosecurity |
| Not researched |
Factual |
8 Mar 2024 |
LeCun said that when the representations from this training process are used as input to a classification system, it works, and he gave this as how we know the process produces a good compressed representation of visual reality.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Language does not enter a child's learning until about a year into life.
Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Factual |
8 Mar 2024 |
The learning techniques Meta's FAIR uses for its JEPA-style joint embedding models are non-contrastive.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Yann LeCun has made the point about AI bias on many social networks and in various forums multiple times.
Factual · 8 Mar 2024 |
Their own work or organisation |
| Not researched |
Factual |
8 Mar 2024 |
In autoregressive LLMs, each generated token carries some probability of taking the output outside the set of reasonable answers.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Yann LeCun co-founded the International Conference on Learning Representations about 14 years before 2024.
Factual · 8 Mar 2024 |
Field building and careers |
| Not researched |
Factual |
8 Mar 2024 |
The Ottoman Empire banned the printing press for 200 years, only for Arabic-language printing.
Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Factual |
8 Mar 2024 |
Meta's business model is to offer services financed either through ads or through business customers.
Factual · 8 Mar 2024 |
Economics and markets |
| Not researched |
Factual |
8 Mar 2024 |
As of March 2024, LLMs with vision extensions don't really understand intuitive physics.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Large language models are trained purely from text.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Government and military organizations are not particularly innovative in AI and are far behind industry.
Factual · 8 Mar 2024 |
AI policy and governance |
| Not researched |
Factual |
8 Mar 2024 |
With contrastive methods, if the output space is large, the number of contrasting samples needed is gigantic.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
LeCun said that LLMs can be thought of as doing a very simple form of optimization in the discrete space of possible token sequences.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
LLMs predict tokens, which are subword units, rather than whole words.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
In the last three to four years, non-contrastive self-supervised methods that do not require negative contrastive samples have been developed.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Through sensory input humans take in far more information than through language (about 10^15 bytes for a four-year-old versus 2x10^13 bytes for 170,000 years of reading).
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
If token errors are independent, the probability of staying within correct answers decreases exponentially with each token produced.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
LeCun's team could not get pixel-predicting generative models to learn good representations of images.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
DINO and I-JEPA were developed at FAIR and work well for learning image representations.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
As of March 2024, nobody knows how to train an AI system to learn multiple levels of representation for hierarchical planning.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Meta developed Wav2Vec, a joint embedding architecture trained with contrastive learning, which achieved success in speech recognition.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
A growing number of studies indicate that LLMs do not help people design or build bioweapons or chemical weapons beyond what search engines and libraries provide.
Factual · 8 Mar 2024 |
Biosecurity |
| Not researched |
Factual |
8 Mar 2024 |
LLaMA 2 has had millions of downloads and thousands of people have contributed ideas to improve it.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Image features learned by self-supervised reconstruction of corrupted images are not good; supervised training with text descriptions gives better recognition performance.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
LLMs don't really have persistent memory.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Current LLMs cannot do deliberate planning using an internal world model (system two).
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
It took decades to fine-tune the design of turbojets so that they were safe, rather than deriving a general safety principle.
Factual · 8 Mar 2024 |
AI safety and alignment |
| Not researched |
Factual |
8 Mar 2024 |
It is not known how to represent distributions over high-dimensional, continuous spaces in useful ways.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
LLMs can't really reason and certainly can't plan.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
LLMs implicitly prevent collapse by giving low probability to other words when a word gets high probability, since probabilities sum to one.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Text is discrete, whereas video is high-dimensional and continuous.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
The proportion of all possible prompts used in LLM training is a tiny subset.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
There is some research on AI planning of tool-use actions, but not a huge amount.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Two-level hierarchical planning can be done in AI when the two levels are designed by hand.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
The computation an LLM spends per produced token is constant.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Yann LeCun published a paper on contrastive methods in 1993.
Factual · 8 Mar 2024 |
Their own work or organisation |
| Not researched |
Factual |
8 Mar 2024 |
In March 2024, LeCun said he did not think there was yet a good solution for AI planning of tool-use action sequences.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
LLMs behave properly on trained prompts, but a prompt outside the training set can break them and produce nonsense.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
V-JEPA is Meta's first system to learn video representations good enough for a supervised classifier head to identify actions in video with pretty good accuracy.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
The Ottoman Empire banned the printing press in part to protect the powerful guild of scribes.
Factual · 8 Mar 2024 |
Technology |
| Not researched |
Factual |
8 Mar 2024 |
The printing press enabled philosophy, rationalism, escape from religious doctrine, democracy and science.
Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Factual |
8 Mar 2024 |
Everything humans learn in the first few years of life, and everything animals learn, has nothing to do with language.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
28 Jun 2023 |
Data center power consumption barely changes because it is largely determined by economics: Google and Meta cannot spend more than a few dollars per user per year on power.
Factual · 28 Jun 2023 |
Economics and markets |
| Not researched |
Factual |
22 Jun 2023 |
A large number of Americans believe in conspiracy theories, though the proportion has not increased over time.
Factual · 22 Jun 2023 |
Philosophy, ethics and society |
| Not researched |
Factual |
22 Jun 2023 |
Yann LeCun gave an opening statement at the Munk Debate on the existential risk of AI on 2023-06-22.
Factual · 22 Jun 2023 |
AI safety and alignment |
| Not researched |
Factual |
18 Jun 2023 |
Meta AI / FAIR has a large presence at CVPR 2023.
Factual · 18 Jun 2023 |
Their own work or organisation |
| Not researched |
Factual |
18 Jun 2023 |
Yann LeCun is an advisor to Enhance Lab.
Factual · 18 Jun 2023 |
Their own work or organisation |
| Not researched |
Factual |
15 Jun 2023 |
Yann LeCun expressed opinions on the future of AI at the Innovation Press Day held at Meta-Paris on Tuesday 2023-06-13.
Factual · 15 Jun 2023 |
Their own work or organisation |
| Not researched |
Factual |
14 Jun 2023 |
Ania Dawid transcribed Yann LeCun's lectures on Energy-Based Models, which he gave at the Les Houches summer school in 2022.
Factual · 14 Jun 2023 |
Science and research methods |
| Not researched |
Factual |
Jun 2023 |
The denoising autoencoder (ref. 51) is a type of contrastive energy-based model.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
Supervised and reinforcement learning require an enormous number of labeled samples or trials.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
Energy-based models can represent multimodal dependencies in which multiple values of y are compatible with a given x.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
Contrastive methods for training energy-based models tend to become very inefficient in high dimensions.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Citation |
Jun 2023 |
Convolutional sparse autoencoders learn features corresponding to known image filters used for feature extraction and compressed representation (ref. 47).
Citation · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
Babies learn almost entirely by observation.
Factual · Jun 2023 |
Science and research methods |
| Not researched |
Factual |
Jun 2023 |
Hopfield networks, fully-connected recurrent networks, were popularized by John Hopfield in 1982.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
Joint embedding predictive architectures (JEPAs) are a new architecture proposed by LeCun in Ref. [6].
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Citation |
Jun 2023 |
Normalizing scores in energy-based models runs into the label bias problem (ref. 39), where transition probabilities out of a state are normalized per state rather than globally.
Citation · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
Supervised and reinforcement learning dominate modern real-world applications and are highly inefficient compared to human learning.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
SSL for images produces blurry predictions.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Citation |
Jun 2023 |
Score matching (ref. 46) is a regularization technique that minimizes the gradient and maximizes the curvature of the energy landscape around data points.
Citation · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
Most large-scale machine learning applications today rely on supervised learning or deep reinforcement learning, which require extreme computational expense.
Factual · Jun 2023 |
AI compute, hardware and data |
| Not researched |
Factual |
Jun 2023 |
Boltzmann machines were fashionable for some time, but the large cost of sampling makes them less useful for real-world applications.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
As of mid-2023, there are no Level 5 self-driving cars, domestic robots, or virtual assistants that learn reliable world models, reason and plan complex action sequences.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
For energy-based models with a continuous y domain, the energy function is smooth and differentiable, allowing gradient-based inference.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
The lecture notes summarize the main ideas of the autonomous intelligence architecture proposed by Yann LeCun.
Factual · Jun 2023 |
Their own work or organisation |