| Not checkable |
Not checkable |
2021 |
I think the big challenge of AI for the next decade,
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
If we had as many world models as there are situations we encounter, then we could do all of them simultaneously.
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
Orangutans are solitary animals that do not seek the company of others.
Factual · 2021 |
Other |
| Not checkable |
Not checkable |
2021 |
Your model of the world should have your actions as one of the inputs, and that will drive you to learn causal models of the world
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And that's the answer to this probably is self supervised running.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
The essence of intelligence is the ability to predict
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And they are extremely inefficient.
Not checkable · 2021 |
|
| Mostly true |
Factual |
2021 |
A cat has fewer than 1 billion neurons, while a dog has 2 billion.
Factual · 2021 |
Other |
| Not checkable |
Not checkable |
2021 |
There's nothing wrong with that.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And the main issue with learning world models or learning predictive models is that the prediction is never a single thing because the…
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
Babies learn an enormous amount of background knowledge about the world that may be the basis of common sense.
Factual · 2021 |
Philosophy, ethics and society |
| Not checkable |
Not checkable |
2021 |
That engine is configurable to the situation at hand.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
The first one is, you know, getting machines to learn to represent the world.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think most people don't actually realize what death is, right?
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
It doesn't make the process exist.
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
Self-supervised learning offers far more training signal than supervised or reinforcement learning.
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
Now, that said, this ability to learn world models, I think is the key to the possibility of running machines that can also reason.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
But I don't think they're fundamental, you know, active learning, even to some extent reinforcement learning.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
It's, to me, it seems very implausible
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
This is the kind of problem we've evolved to solve.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So it's the more fundamental question to ask.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
It's just statistics of a particular kind.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So I think the action part would be necessary for having causal models of the world.
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
Generally, the model is mostly deterministic and mostly built by hand, though sometimes a few parameters have to be identified through what is called systems identification.
Factual · 2021 |
Technology |
| Not researched |
Factual |
2021 |
Barlow Twins is a method for maximizing the information content of a vector.
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
And it might suggest the idea that consciousness basically is the module that configures this world model of yours.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
Can we get them to spontaneously learn good hierarchical representations of actions?
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So cats and rats are incredibly curious.
Not checkable · 2021 |
|
| Mostly true |
Factual |
2021 |
Since the 1990s, algorithms exist that learn oriented edge detectors fully unsupervised with the equivalent of a few minutes of real time.
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
consciousness is not a consequence of the power of our mind,
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
The first vision systems in animals evolved mainly for localization, with very little recognition.
Factual · 2021 |
Science and research methods |
| Unresolvable |
Prediction |
2021 |
A machine cannot be trained to be intelligent purely from text.
Prediction · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
a lot of the things that are said about consciousness remind me of the questions people were asking themselves in the 18th century or 17th…
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
is how do we get machines to learn predictive models of the world that deal with uncertainty
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I don't even know how to measure an answer to that question.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So whether this will take us all the way to human-level intelligence or something, or just cat-level intelligence is not clear, but among…
Not checkable · 2021 |
|
| Unresolvable |
Factual |
2021 |
The knowledge needed for house-cat intelligence fits in about 800 million neurons.
Factual · 2021 |
Other |
| Mostly true |
Factual |
2021 |
Self-supervised training of visual representations from video has not yet been successful.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
Representations from self-supervised methods that eliminate augmentation distortions cannot be used for object detection and localization because that information is lost.
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
So that suggests you only have one model in your head.
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
Yann LeCun proposes self-supervised learning as the way to get machines to learn to represent the world.
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
I think those things are actually very simple to learn.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think data augmentation is a temporary necessary evil.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think we give way too much importance to language as a substrate of intelligence as humans
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
It sounds very dramatic and very impressive, but it probably isn't mechanistically speaking.
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
A sea change in favor of convolutional nets occurred when datasets became bigger and GPUs became available.
Factual · 2021 |
AI compute, hardware and data |
| Not checkable |
Not checkable |
2021 |
The second reason it may be necessary or at least more efficient is that active learning basically goes for the juggler of what you don't…
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So those things have to be learned.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I'm actually not very positive about
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
But we judge each other based on our ability
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I don't get into those discussions because they are kind of pointless.
Not checkable · 2021 |
|
| Resolved true |
Factual |
2021 |
The American financial payment system is much more lax about signature verification than Europe's.
Factual · 2021 |
Economics and markets |
| Not checkable |
Not checkable |
2021 |
And the third one is something we have no idea how to solve.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
Nature has not made us good at chess.
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
Until very recently there were no working methods for training image representations by masking parts of images and reconstructing them.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
Oriented edge detectors can be learned within minutes of opening one's eyes.
Factual · 2021 |
Science and research methods |
| Not researched |
Factual |
2021 |
LeCun said he believes no text in the world explains the physical situation he was describing (an object that is "sitting on" something else).
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
But fundamentally, it's still the same thing in building up the intuition about how to, in a self-supprised way to construct background…
Not checkable · 2021 |
|
| Unresolvable |
Factual |
2021 |
The most intelligent AI systems have less common sense than a house cat.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
In the early 1990s LeCun and Bell Labs colleagues including Jane Bromley proposed using negative example pairs, now called contrastive learning.
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
So even that can be learned.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
But because we have only one world model, we have to be conscious.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I know you're not gonna just dematerialize and reappear three meters down the hall, because I know what's possible and what's impossible…
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
The critic that makes long-term prediction of the outcome, which is the eventual result of this, that's learned.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
then you can use gradient-based type methods to do planning,
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And obviously we learn them.
Not checkable · 2021 |
|
| Unresolvable |
Factual |
2021 |
At that earlier time, datasets were so small that hand-engineered methods worked better than convolutional nets.
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
We basically cannot attend to anything else.
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
LeCun revived Hinton's idea together with his postdocs at FAIR, producing a method called Barlow Twins.
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
Your own brain computes whether you're happy or not, right?
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think almost all of it is learned through a sort
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
Model predictive control in classical optimal control has existed since the early 1960s.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
The distinguishing aspects of human intelligence appeared only in the last million years.
Factual · 2021 |
Science and research methods |
| Not checkable |
Not checkable |
2021 |
I think it's basically hopeless.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And reinforcement learning requires a ridiculously large number of trial and errors for a system to run anything.
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
Without specific provisions, a Siamese network trained on similar pairs ends up ignoring its input and producing a constant output vector.
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
And it's not clear to me how you would do this in a machine.
Not checkable · 2021 |
|
| Mostly true |
Factual |
2021 |
In reinforcement learning the machine receives only a single scalar feedback, and only occasionally.
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
a lot of the things you need for survival are localization and detection.
Not checkable · 2021 |
|
| Unresolvable |
Prediction |
2021 |
All of intelligence could possibly be solved by filling in blanks in language and similar self-supervised prediction.
Prediction · 2021 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
2021 |
It can still find the object in the image.
Not checkable · 2021 |
|
| Unresolvable |
Factual |
2021 |
AI is not anywhere close to reproducing the learning done by a cat brain of about 800 million neurons.
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
I mean, we're just bad.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think humans use that relatively rarely
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
Zeroth-order, gradient-free black-box optimization is much less efficient than gradient-based optimization.
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
So, there's a huge independence hypothesis in this process, which is not actually true.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And if you know that you don't know, then it makes you curious.
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
Self-supervised pre-training has been less successful in images than in language, is making progress, and relies on manual data augmentation.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
LeCun said he thinks the entire human prefrontal cortex is basically the engine for a world model.
Factual · 2021 |
Science and research methods |
| Not checkable |
Not checkable |
2021 |
It's called an IQ test.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So I think this ability to construct models of the world is basically the essence, the essence of intelligence
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think a lot of those things are learned because they are easy to learn.
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
In supervised learning the machine is given a few bits of feedback per sample.
Factual · 2021 |
AI capabilities |
| Not checkable |
Not checkable |
2021 |
Okay. I don't know what consciousness is
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
We completely suck at chess.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I get wiped out in 10 plays, right?
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So the type of data augmentation you need to do depends on the task you want eventually the system to solve
Not checkable · 2021 |
|
| Not researched |
Factual |
2021 |
A paper coming out of the FAIR group at Menlo Park shows a masking-based method that works very well, without data augmentation.
Factual · 2021 |
Their own work or organisation |
| Not researched |
Factual |
2021 |
Lions attack bigger animals than cheetahs do.
Factual · 2021 |
Other |
| Not checkable |
Not checkable |
2021 |
It's basically planning, and a lot of planning systems in robotics are actually based on this,
Not checkable · 2021 |
|