| Not researched |
Factual |
8 Mar 2024 |
Attempts to train systems to predict video pixels (including latent-variable approaches) have been a complete failure essentially.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
LLMs do not plan their answers before producing them; they produce one word after another.
Factual · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
We’re really fooled by it.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
There are now about six different non-contrastive methods for preventing collapse in joint embedding training.
Factual · 8 Mar 2024 |
AI capabilities |
| Not yet due |
Prediction |
8 Mar 2024 |
He thinks the next decade will be really interesting for robots. He said the robotics industry has been waiting 10 to 20 years to emerge without really doing so, apart from pre-programmed behaviour.
Prediction · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
Well, there is a natural fear of new technology and the impact it can have in society.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
I-JEPA does not need to know the input is an image, since it only uses masking, whereas DINO relies on image-specific transformations.
Factual · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
you cannot compare two things as to whether one is more intelligent than the other.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
it’s because bias is in the eye of the beholder.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
In the first few months of life, babies have little influence on the world and can only observe.
Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Unresolvable |
Prediction |
8 Mar 2024 |
Keeping AI systems locked up for security reasons would lead to a future in which people's information diet is controlled by a small number of companies with proprietary systems.
Prediction · 8 Mar 2024 |
AI policy and governance |
| Not checkable |
Not checkable |
8 Mar 2024 |
Then the fine-tuning that people will add or the additional guardrails that people will add will cater to their community, whatever it is.
Not checkable · 8 Mar 2024 |
|
| Mostly true |
Factual |
8 Mar 2024 |
Rocket trajectories have been planned using model-based optimization of command sequences since computers existed, essentially since the early 1960s.
Factual · 8 Mar 2024 |
Other |
| Not checkable |
Not checkable |
8 Mar 2024 |
The system cannot predict what’s going to be in the room as the camera is panning.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
intrinsically and even social networks are a good thing if you run them properly.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
The printing press transformed the world by making people smarter and better informed.
Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not researched |
Factual |
8 Mar 2024 |
One set of non-contrastive techniques is based on distillation, including DeepMind's BYOL and FAIR's I-JEPA.
Factual · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
So learning representation is really the thing.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
Researchers currently have no idea how to do learned hierarchical planning in AI.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
LLMs with vision extensions are not trained on video.
Factual · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
History of the world is whenever there is a progress someplace, there is a countermeasure and it’s a cat and mouse game.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
A joint embedding system trained only to match representations of an image and its corrupted version collapses, ignoring the input and producing constant representations.
Factual · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
if you want those systems to be useful, they will necessarily have to offend a number of people inevitably.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
Gradient-based optimization can only be done in continuous spaces with differentiable functions.
Factual · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
Again, I think the answer to this is open source platforms and then enabling a widely diverse set of people to build AI assistance that…
Not checkable · 8 Mar 2024 |
|
| Not yet due |
Prediction |
8 Mar 2024 |
LeCun said he thinks that over the next few years there will be many AI systems with planning capability, and that their blueprint will be extremely different from autoregressive LLMs.
Prediction · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
LLMs are not controllable,
Not checkable · 8 Mar 2024 |
|
| Unresolvable |
Factual |
8 Mar 2024 |
Yann LeCun has had much experience working with people who are smarter than him.
Factual · 8 Mar 2024 |
Their own work or organisation |
| Unresolvable |
Factual |
8 Mar 2024 |
Meta's ability to derive revenue from AI technology is not impaired by releasing base models as open source.
Factual · 8 Mar 2024 |
Economics and markets |
| Not researched |
Factual |
8 Mar 2024 |
Actually building a chemical weapon or bioweapon is much harder than obtaining instructions, and an LLM will not help with that step.
Factual · 8 Mar 2024 |
Biosecurity |
| Not yet due |
Prediction |
8 Mar 2024 |
Building AI systems with world models, planning and hierarchical representations like the human brain will take at least a decade, probably much more.
Prediction · 8 Mar 2024 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
8 Mar 2024 |
I love that question.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
Certainly, scale is necessary but not sufficient.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
Because intelligence is the commodity that is most in demand.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
They don’t have language and they understand the world much better than any LLM.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
For about 10 years, LeCun's team tried to learn image and video representations with generative models that predict pixels, and failed.
Factual · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
And what works against this is people who think that for reasons of security, we should keep the AI systems under lock and key because it’s…
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
The professions that are going to be hot 10 or 15 years from now, we have no idea today what they’re going to be.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
No one in the world, including countries, has used bioweapons, and they are banned by international treaties.
Factual · 8 Mar 2024 |
Biosecurity |
| Unresolvable |
Prediction |
8 Mar 2024 |
AI will bring a gradual shift between professions rather than mass unemployment.
Prediction · 8 Mar 2024 |
AI, the economy and work |
| Not checkable |
Not checkable |
8 Mar 2024 |
It’s the same difference as the difference between what psychologists call system one and system two in humans
Not checkable · 8 Mar 2024 |
|
| Unresolvable |
Prediction |
8 Mar 2024 |
Designing AI guardrails will not be solved by a single silver bullet or a mathematical proof of safety.
Prediction · 8 Mar 2024 |
AI safety and alignment |
| Not researched |
Factual |
8 Mar 2024 |
LeCun said that when the representations from this training process are used as input to a classification system, it works, and he gave this as how we know the process produces a good compressed representation of visual reality.
Factual · 8 Mar 2024 |
AI capabilities |
| Unresolvable |
Prediction |
8 Mar 2024 |
Progress toward advanced AI will be continuous rather than a single event.
Prediction · 8 Mar 2024 |
AI timelines and forecasting |
| Not researched |
Factual |
8 Mar 2024 |
Language does not enter a child's learning until about a year into life.
Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Not checkable |
Not checkable |
8 Mar 2024 |
They basically are going to have to regurgitate the template that they’ve been trained on.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
We’re basically cheating. We’re using language as a crutch to help the deficiencies of our vision systems to learn good representations…
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
I knew they were wrong when they were saying it.
Not checkable · 8 Mar 2024 |
|
| Unresolvable |
Prediction |
8 Mar 2024 |
Future AI systems will not have a desire to dominate, because that desire would have to be hardwired into them.
Prediction · 8 Mar 2024 |
AI safety and alignment |
| Not checkable |
Not checkable |
8 Mar 2024 |
That is not to say that autoregressive LLMs are not useful.
Not checkable · 8 Mar 2024 |
|
| Unresolvable |
Prediction |
8 Mar 2024 |
Advanced AI systems will not become a species that competes with humans.
Prediction · 8 Mar 2024 |
AI safety and alignment |
| Not checkable |
Not checkable |
8 Mar 2024 |
Well, so no, you will have to build a specific architecture to allow for hierarchical planning.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
The learning techniques Meta's FAIR uses for its JEPA-style joint embedding models are non-contrastive.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Yann LeCun has made the point about AI bias on many social networks and in various forums multiple times.
Factual · 8 Mar 2024 |
Their own work or organisation |
| Not checkable |
Not checkable |
8 Mar 2024 |
is a lot of underlying reality, which is not expressed in language.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
So that’s what we’re missing from current AI systems.
Not checkable · 8 Mar 2024 |
|
| Mixed |
Factual |
8 Mar 2024 |
For the last 12 to 15 years, people have claimed AGI is just around the corner and have been systematically wrong.
Factual · 8 Mar 2024 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
8 Mar 2024 |
If you’re really interested in human level AI, abandon the idea of generative AI.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
And in language, we can get away without doing this because language is already to some level abstract and already has eliminated a lot of…
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
But if you want to go down to how you climb down the stairs or just stand up from your chair in terms of words, you can’t do it.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
When you and I talk, you and I are bilingual, we think about what we’re going to say, and it’s relatively independent of the language in…
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
Language is a very approximate representation or percepts and/or mental models.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
In autoregressive LLMs, each generated token carries some probability of taking the output outside the set of reasonable answers.
Factual · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
We just assume that if a system is fluent in manipulating language, then it has all the characteristics of human intelligence, but that…
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
Yann LeCun co-founded the International Conference on Learning Representations about 14 years before 2024.
Factual · 8 Mar 2024 |
Field building and careers |
| Not checkable |
Not checkable |
8 Mar 2024 |
Well, not really, only in a very simple way.
Not checkable · 8 Mar 2024 |
|
| Unresolvable |
Prediction |
8 Mar 2024 |
Millions of humanoid robots will eventually be walking around, though not soon.
Prediction · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
Yes.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
The Ottoman Empire banned the printing press for 200 years, only for Arabic-language printing.
Factual · 8 Mar 2024 |
Philosophy, ethics and society |
| Unresolvable |
Prediction |
8 Mar 2024 |
In future, people will interact with persuasive bad actors through their own AI assistants, which will be as smart as the actors' systems.
Prediction · 8 Mar 2024 |
AI safety and alignment |
| Unresolvable |
Prediction |
8 Mar 2024 |
LeCun said there is ample evidence that generative models will not be able to learn good representations of the real world.
Prediction · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
So it’s going to be impossible to produce systems that are unbiased for everyone.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
Meta's business model is to offer services financed either through ads or through business customers.
Factual · 8 Mar 2024 |
Economics and markets |
| Not researched |
Factual |
8 Mar 2024 |
As of March 2024, LLMs with vision extensions don't really understand intuitive physics.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
Large language models are trained purely from text.
Factual · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
I haven’t been that excited about the direction of machine learning and AI since 10 years ago when Fairway was started.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
I just think that most of the information of this type that we have accumulated when we were babies, it’s just not present in text, in any…
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
Government and military organizations are not particularly innovative in AI and are far behind industry.
Factual · 8 Mar 2024 |
AI policy and governance |
| Not checkable |
Not checkable |
8 Mar 2024 |
So it’s much better to do an optimization in continuous space where you can do gradient and descent as opposed to generate tons of things…
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
So LLMs or vision systems that are trained by reconstruction generate the inputs.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
So I share that feeling, I think people are fundamentally good.
Not checkable · 8 Mar 2024 |
|
| Unresolvable |
Prediction |
8 Mar 2024 |
Without world models that let systems learn how the world works, there will be no significant progress in robotics.
Prediction · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
I thought it was because the rulers just wanted to preserve the control over the population and the religious dogma and everything.
Not checkable · 8 Mar 2024 |
|
| Mostly true |
Factual |
8 Mar 2024 |
Current LLMs can do none of understanding the physical world, persistent memory, reasoning and planning, or only in a very primitive way.
Factual · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
we cannot afford those systems to come from a handful of companies on the west coast of the US because those systems will constitute the…
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
With contrastive methods, if the output space is large, the number of contrasting samples needed is gigantic.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
LeCun said that LLMs can be thought of as doing a very simple form of optimization in the discrete space of possible token sequences.
Factual · 8 Mar 2024 |
AI capabilities |
| Unresolvable |
Prediction |
8 Mar 2024 |
AI safety guardrails will be developed through a progressive, iterative design process.
Prediction · 8 Mar 2024 |
AI safety and alignment |
| Not checkable |
Not checkable |
8 Mar 2024 |
If we eventually have systems that are built with this blueprint, we can put guardrails in those systems that guarantee that there is a…
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
But objectively-driven AI, so systems that derive their answers by optimization of an objective means they have to optimize this objective…
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
I see the danger of this concentration of power through proprietary AI systems as a much bigger danger than everything else.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
So it’s essentially impossible for systems like this that can inevitably formulate political opinions, and opinions about various things…
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
And so the proper way to train a system is to first have it learn good representations of the world and world models from mostly…
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
And some of that work has been incredibly successful.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
It doesn’t think about its answer really. It retrieves it because it’s accumulated a lot of knowledge.
Not checkable · 8 Mar 2024 |
|
| Unresolvable |
Prediction |
8 Mar 2024 |
Training on large data sets may turn out to be necessary for world-model systems to have emergent properties like LLMs.
Prediction · 8 Mar 2024 |
AI capabilities |
| Not checkable |
Not checkable |
8 Mar 2024 |
So the only way you’re going to have an AI industry, the only way you’re going to have AI systems that are not uniquely biased is if you…
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
I think it should not be abandoned completely, but I think it’s use should be minimized because it’s incredibly inefficient in terms of…
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
LLMs predict tokens, which are subword units, rather than whole words.
Factual · 8 Mar 2024 |
AI capabilities |
| Not researched |
Factual |
8 Mar 2024 |
In the last three to four years, non-contrastive self-supervised methods that do not require negative contrastive samples have been developed.
Factual · 8 Mar 2024 |
AI capabilities |