| Not researched |
Sam Altman | Factual |
18 Mar 2024 |
The OpenAI board had the legal ability to fire Sam Altman but in practice the firing did not work.
Sam Altman · Factual · 18 Mar 2024 |
Their own work or organisation |
| Not researched |
Sam Altman | Factual |
18 Mar 2024 |
OpenAI uses lots of human data in its work.
Sam Altman · Factual · 18 Mar 2024 |
Their own work or organisation |
| Not researched |
Yann LeCun | Factual |
14 Mar 2024 |
As of March 2024 there is no fully autonomous, reliable self-driving, even though millions of hours of labeled training data exist.
Yann LeCun · Factual · 14 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
14 Mar 2024 |
The report of the Committee on AI appointed by the French government is available for download, with a summary in English.
Yann LeCun · Factual · 14 Mar 2024 |
AI policy and governance |
| Not researched |
Yann LeCun | Factual |
14 Mar 2024 |
Yann LeCun is a member of the Committee on AI appointed by the French government.
Yann LeCun · Factual · 14 Mar 2024 |
AI policy and governance |
| Not researched |
Yann LeCun | Factual |
13 Mar 2024 |
The French AI Commission's report mentions the importance of open models.
Yann LeCun · Factual · 13 Mar 2024 |
AI policy and governance |
| Not researched |
Yann LeCun | Factual |
13 Mar 2024 |
The French Commission on Artificial Intelligence delivered a report to the President of the Republic containing 25 recommendations, 7 of them priority recommendations.
Yann LeCun · Factual · 13 Mar 2024 |
AI policy and governance |
| Not researched |
Yann LeCun | Factual |
10 Mar 2024 |
The adaptive equalizer is basically a linear classifier trained with least squares, a direct descendant of Bernie Widrow's Adaline, a competitor of the Perceptron.
Yann LeCun · Factual · 10 Mar 2024 |
Technology |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Wav2Vec-based systems can produce multilingual speech recognition using mostly unlabeled data and only a few minutes of labeled data.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Humans spend more time solving complex problems or questions because they are more difficult.
Yann LeCun · Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Throughout history, technological revolutions and cultural phenomena were accompanied by media reactions blaming society's problems on the change.
Yann LeCun · Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Asked how this is done for visual data, LeCun said that he and his colleagues have been doing it with I-JEPA architectures.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
RLHF reward models are used only for training LLMs, not for inference.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Human feedback is what had the transformational effect on large language models, and it can be used in many ways, some purely supervised.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Meta published its V-JEPA work the week before this interview, as a first step toward training AI systems on video.
Yann LeCun · Factual · 8 Mar 2024 |
Their own work or organisation |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
After abandoning pixel prediction, joint embedding with prediction in representation space worked for learning representations.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
The success of the US tech industry and Silicon Valley is due to information circulating and disseminating very quickly.
Yann LeCun · Factual · 8 Mar 2024 |
Economics and markets |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
The Catholic Church disliked the printing press but had no choice but to accept it.
Yann LeCun · Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Most of what humans learn and know comes from observation and interaction with the real world, not from language.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Attempts to train systems to predict video pixels (including latent-variable approaches) have been a complete failure essentially.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
LLMs do not plan their answers before producing them; they produce one word after another.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
There are now about six different non-contrastive methods for preventing collapse in joint embedding training.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
I-JEPA does not need to know the input is an image, since it only uses masking, whereas DINO relies on image-specific transformations.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
In the first few months of life, babies have little influence on the world and can only observe.
Yann LeCun · Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
The printing press transformed the world by making people smarter and better informed.
Yann LeCun · Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
One set of non-contrastive techniques is based on distillation, including DeepMind's BYOL and FAIR's I-JEPA.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Researchers currently have no idea how to do learned hierarchical planning in AI.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
LLMs with vision extensions are not trained on video.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
A joint embedding system trained only to match representations of an image and its corrupted version collapses, ignoring the input and producing constant representations.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Gradient-based optimization can only be done in continuous spaces with differentiable functions.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Actually building a chemical weapon or bioweapon is much harder than obtaining instructions, and an LLM will not help with that step.
Yann LeCun · Factual · 8 Mar 2024 |
Biosecurity |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
For about 10 years, LeCun's team tried to learn image and video representations with generative models that predict pixels, and failed.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
No one in the world, including countries, has used bioweapons, and they are banned by international treaties.
Yann LeCun · Factual · 8 Mar 2024 |
Biosecurity |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
LeCun said that when the representations from this training process are used as input to a classification system, it works, and he gave this as how we know the process produces a good compressed representation of visual reality.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Language does not enter a child's learning until about a year into life.
Yann LeCun · Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
The learning techniques Meta's FAIR uses for its JEPA-style joint embedding models are non-contrastive.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Yann LeCun has made the point about AI bias on many social networks and in various forums multiple times.
Yann LeCun · Factual · 8 Mar 2024 |
Their own work or organisation |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
In autoregressive LLMs, each generated token carries some probability of taking the output outside the set of reasonable answers.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Yann LeCun co-founded the International Conference on Learning Representations about 14 years before 2024.
Yann LeCun · Factual · 8 Mar 2024 |
Field building and careers |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
The Ottoman Empire banned the printing press for 200 years, only for Arabic-language printing.
Yann LeCun · Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Meta's business model is to offer services financed either through ads or through business customers.
Yann LeCun · Factual · 8 Mar 2024 |
Economics and markets |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
As of March 2024, LLMs with vision extensions don't really understand intuitive physics.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Large language models are trained purely from text.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Government and military organizations are not particularly innovative in AI and are far behind industry.
Yann LeCun · Factual · 8 Mar 2024 |
AI policy and governance |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
With contrastive methods, if the output space is large, the number of contrasting samples needed is gigantic.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
LeCun said that LLMs can be thought of as doing a very simple form of optimization in the discrete space of possible token sequences.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
LLMs predict tokens, which are subword units, rather than whole words.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
In the last three to four years, non-contrastive self-supervised methods that do not require negative contrastive samples have been developed.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Through sensory input humans take in far more information than through language (about 10^15 bytes for a four-year-old versus 2x10^13 bytes for 170,000 years of reading).
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
If token errors are independent, the probability of staying within correct answers decreases exponentially with each token produced.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
LeCun's team could not get pixel-predicting generative models to learn good representations of images.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
DINO and I-JEPA were developed at FAIR and work well for learning image representations.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
As of March 2024, nobody knows how to train an AI system to learn multiple levels of representation for hierarchical planning.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Meta developed Wav2Vec, a joint embedding architecture trained with contrastive learning, which achieved success in speech recognition.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
A growing number of studies indicate that LLMs do not help people design or build bioweapons or chemical weapons beyond what search engines and libraries provide.
Yann LeCun · Factual · 8 Mar 2024 |
Biosecurity |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
LLaMA 2 has had millions of downloads and thousands of people have contributed ideas to improve it.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Image features learned by self-supervised reconstruction of corrupted images are not good; supervised training with text descriptions gives better recognition performance.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
LLMs don't really have persistent memory.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Current LLMs cannot do deliberate planning using an internal world model (system two).
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
It took decades to fine-tune the design of turbojets so that they were safe, rather than deriving a general safety principle.
Yann LeCun · Factual · 8 Mar 2024 |
AI safety and alignment |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
It is not known how to represent distributions over high-dimensional, continuous spaces in useful ways.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
LLMs can't really reason and certainly can't plan.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
LLMs implicitly prevent collapse by giving low probability to other words when a word gets high probability, since probabilities sum to one.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Text is discrete, whereas video is high-dimensional and continuous.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
The proportion of all possible prompts used in LLM training is a tiny subset.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
There is some research on AI planning of tool-use actions, but not a huge amount.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Two-level hierarchical planning can be done in AI when the two levels are designed by hand.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
The computation an LLM spends per produced token is constant.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Yann LeCun published a paper on contrastive methods in 1993.
Yann LeCun · Factual · 8 Mar 2024 |
Their own work or organisation |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
In March 2024, LeCun said he did not think there was yet a good solution for AI planning of tool-use action sequences.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
LLMs behave properly on trained prompts, but a prompt outside the training set can break them and produce nonsense.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
V-JEPA is Meta's first system to learn video representations good enough for a supervised classifier head to identify actions in video with pretty good accuracy.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
The Ottoman Empire banned the printing press in part to protect the powerful guild of scribes.
Yann LeCun · Factual · 8 Mar 2024 |
Technology |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
The printing press enabled philosophy, rationalism, escape from religious doctrine, democracy and science.
Yann LeCun · Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Yann LeCun | Factual |
8 Mar 2024 |
Everything humans learn in the first few years of life, and everything animals learn, has nothing to do with language.
Yann LeCun · Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Demis Hassabis | Factual |
28 Feb 2024 |
DeepMind's pioneering work, inspired by neuroscience, included experience replay and the notion of attention.
Demis Hassabis · Factual · 28 Feb 2024 |
Their own work or organisation |
| Not researched |
Demis Hassabis | Factual |
28 Feb 2024 |
In 2010, no one was thinking about AI, let alone AGI.
Demis Hassabis · Factual · 28 Feb 2024 |
Their own work or organisation |
| Not researched |
Demis Hassabis | Factual |
28 Feb 2024 |
Since DeepMind's founding, it has said that it does not have to wait for AGI to bring incredible benefits to the world.
Demis Hassabis · Factual · 28 Feb 2024 |
Their own work or organisation |
| Not researched |
Demis Hassabis | Factual |
28 Feb 2024 |
DeepMind joined forces with Google in 2014.
Demis Hassabis · Factual · 28 Feb 2024 |
Their own work or organisation |
| Not researched |
Demis Hassabis | Factual |
28 Feb 2024 |
When large AI models improve at coding, their general reasoning can also improve.
Demis Hassabis · Factual · 28 Feb 2024 |
AI capabilities |
| Not researched |
Demis Hassabis | Factual |
28 Feb 2024 |
RL agents like AlphaZero and AlphaGo are active learners whose decisions determine what data or experience they get next.
Demis Hassabis · Factual · 28 Feb 2024 |
AI capabilities |
| Not researched |
Demis Hassabis | Factual |
28 Feb 2024 |
Downstream capabilities such as MMLU or math performance sometimes don't follow from predictable improvements in training loss.
Demis Hassabis · Factual · 28 Feb 2024 |
Science and research methods |
| Not researched |
Demis Hassabis | Factual |
28 Feb 2024 |
Google DeepMind is still carrying out robotics research.
Demis Hassabis · Factual · 28 Feb 2024 |
Their own work or organisation |
| Not researched |
Demis Hassabis | Factual |
28 Feb 2024 |
Google DeepMind and Google have published thousands of papers, including AlphaFold, transformers and AlphaGo.
Demis Hassabis · Factual · 28 Feb 2024 |
Their own work or organisation |
| Not researched |
Demis Hassabis | Factual |
28 Feb 2024 |
AlphaZero and AlphaGo may look at around tens of thousands of possible positions to decide their next move.
Demis Hassabis · Factual · 28 Feb 2024 |
AI capabilities |
| Not researched |
Demis Hassabis | Factual |
28 Feb 2024 |
Scaling laws require scaling up hyperparameters and new innovations at each new scale, not just repeating the same recipe.
Demis Hassabis · Factual · 28 Feb 2024 |
Science and research methods |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
Tesla is pursuing a dual path for AI training using both Nvidia hardware and its own Dojo supercomputer.
Elon Musk · Factual · 24 Jan 2024 |
AI compute, hardware and data |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
Tesla's in-car inference hardware is now Hardware 4, which is the second version of the Tesla-designed AI inference chip.
Elon Musk · Factual · 24 Jan 2024 |
AI compute, hardware and data |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
Building Tesla's next-generation vehicle requires many specialized machines.
Elon Musk · Factual · 24 Jan 2024 |
Technology |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
Tesla's Dojo supercomputer is working and running training jobs, and Tesla is scaling it up.
Elon Musk · Factual · 24 Jan 2024 |
AI compute, hardware and data |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
Tesla is now on what it calls Hardware 4, which Musk describes as version 2 of the Tesla-designed AI inference chip.
Elon Musk · Factual · 24 Jan 2024 |
AI compute, hardware and data |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
Tesla's market share in Japan is remarkably low.
Elon Musk · Factual · 24 Jan 2024 |
Economics and markets |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
Tesla replaced 330,000 lines of C++ code with neural nets in FSD version 12.
Elon Musk · Factual · 24 Jan 2024 |
AI capabilities |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
Tesla's market share in Japan is lower than that of non-Japanese carmakers such as Mercedes or BMW.
Elon Musk · Factual · 24 Jan 2024 |
Economics and markets |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
The situation Musk described is very much constrained by production, not by demand.
Elon Musk · Factual · 24 Jan 2024 |
Economics and markets |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
Version 1 of Tesla's self-driving chip hardware was Mobileye, version 2 was Nvidia, and version 3 was Tesla-designed.
Elon Musk · Factual · 24 Jan 2024 |
AI compute, hardware and data |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
Model Y was somewhat of a variant on Model 3, which made its production a much easier situation.
Elon Musk · Factual · 24 Jan 2024 |
Their own work or organisation |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
In Q4 2023 Tesla produced vehicles at an annualized run rate of almost 2 million cars a year.
Elon Musk · Factual · 24 Jan 2024 |
Economics and markets |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
Japan is the third-largest national car market in the world.
Elon Musk · Factual · 24 Jan 2024 |
Economics and markets |
| Not researched |
Elon Musk | Factual |
24 Jan 2024 |
FSD version 12 is the first time AI is used for path planning and vehicle controls, not just object perception.
Elon Musk · Factual · 24 Jan 2024 |
AI capabilities |