| Not checkable |
Not checkable |
2021 |
The first one is, you know, getting machines to learn to represent the world.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think most people don't actually realize what death is, right?
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
It doesn't make the process exist.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
Now, that said, this ability to learn world models, I think is the key to the possibility of running machines that can also reason.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
But I don't think they're fundamental, you know, active learning, even to some extent reinforcement learning.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
It's, to me, it seems very implausible
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
This is the kind of problem we've evolved to solve.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So it's the more fundamental question to ask.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
It's just statistics of a particular kind.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So I think the action part would be necessary for having causal models of the world.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And it might suggest the idea that consciousness basically is the module that configures this world model of yours.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
Can we get them to spontaneously learn good hierarchical representations of actions?
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So cats and rats are incredibly curious.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
consciousness is not a consequence of the power of our mind,
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
a lot of the things that are said about consciousness remind me of the questions people were asking themselves in the 18th century or 17th…
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
is how do we get machines to learn predictive models of the world that deal with uncertainty
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I don't even know how to measure an answer to that question.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So whether this will take us all the way to human-level intelligence or something, or just cat-level intelligence is not clear, but among…
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So that suggests you only have one model in your head.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think those things are actually very simple to learn.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think data augmentation is a temporary necessary evil.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think we give way too much importance to language as a substrate of intelligence as humans
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
It sounds very dramatic and very impressive, but it probably isn't mechanistically speaking.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
The second reason it may be necessary or at least more efficient is that active learning basically goes for the juggler of what you don't…
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So those things have to be learned.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I'm actually not very positive about
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
But we judge each other based on our ability
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I don't get into those discussions because they are kind of pointless.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And the third one is something we have no idea how to solve.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
Nature has not made us good at chess.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
But fundamentally, it's still the same thing in building up the intuition about how to, in a self-supprised way to construct background…
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So even that can be learned.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
But because we have only one world model, we have to be conscious.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I know you're not gonna just dematerialize and reappear three meters down the hall, because I know what's possible and what's impossible…
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
The critic that makes long-term prediction of the outcome, which is the eventual result of this, that's learned.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
then you can use gradient-based type methods to do planning,
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And obviously we learn them.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
We basically cannot attend to anything else.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
Your own brain computes whether you're happy or not, right?
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think almost all of it is learned through a sort
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think it's basically hopeless.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And reinforcement learning requires a ridiculously large number of trial and errors for a system to run anything.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And it's not clear to me how you would do this in a machine.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
a lot of the things you need for survival are localization and detection.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
It can still find the object in the image.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I mean, we're just bad.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think humans use that relatively rarely
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So, there's a huge independence hypothesis in this process, which is not actually true.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And if you know that you don't know, then it makes you curious.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
It's called an IQ test.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So I think this ability to construct models of the world is basically the essence, the essence of intelligence
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think a lot of those things are learned because they are easy to learn.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
Okay. I don't know what consciousness is
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
We completely suck at chess.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I get wiped out in 10 plays, right?
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So the type of data augmentation you need to do depends on the task you want eventually the system to solve
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
It's basically planning, and a lot of planning systems in robotics are actually based on this,
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And then the principle of the training procedure is to minimize the difference of the output or the representation between the clean…
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think it's so outside of my radar screen that it's not really
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And we just have no idea how to do this properly.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
Now, what I'm much more enthusiastic about these days is non-contrastive methods.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And it's the thing I'm the most excited about in machine learning in the last 15 years.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
In other words, I believe in grounded intelligence.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So first of all, it's quite possible that intelligence is just statistics.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
It's just not very good at finding the exact boundaries of that object
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So I think at some level of description, intelligence is just statistics, but that doesn't mean you won't have models that have deep…
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And that's why we don't have self-driving cars.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
We haven't answered that question yet.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think that's our best shot at the moment.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think we can go really far.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So I think a lot of what is said about consciousness is of that nature.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
So it could be useful to have that kind of curiosity.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And it's also an annoying step because you need to have that knowledge of what data augmentation you can do that do not change the nature…
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
but we are much better at learning differentiable models to the world.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And the number of continuation grows with the interval of time that you're asking the system to make a prediction for.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
Every time you see a percept, you can perturb it in some way.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
And perception is learned and your model of the world is learned.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
This type of learning is not learning a task.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
2021 |
I think there is a lot of different types of intelligence.
Not checkable · 2021 |
|
| Not checkable |
Not checkable |
29 Sep 2020 |
I also nominate Donald J. Trump to the Fields Medal for proving that the algebraic structure of net worth is a cyclic group: above a…
Not checkable · 29 Sep 2020 |
|
| Not checkable |
Not checkable |
29 Sep 2020 |
I nominate Donald J. Trump to the Nobel Prize in Economics for figuring out how to spend money like a billionaire while having a negative…
Not checkable · 29 Sep 2020 |
|
| Not checkable |
Not checkable |
26 Sep 2020 |
Very cool paper on training Siamese nets to identify whether two states of a physical system belong to the same trajectory or not.
Not checkable · 26 Sep 2020 |
|
| Not checkable |
Not checkable |
26 Sep 2020 |
Good practically-oriented eng school grad: it works, but you don't know why.
Not checkable · 26 Sep 2020 |
|
| Not checkable |
Not checkable |
25 Sep 2020 |
The future of science and technology belongs to young Africans.
Not checkable · 25 Sep 2020 |
|
| Not checkable |
Not checkable |
25 Sep 2020 |
It was a pleasure chatting with you and answering questions from your viewers.
Not checkable · 25 Sep 2020 |
|
| Not checkable |
Not checkable |
24 Sep 2020 |
A good way to evaluate the robustness (or brittleness) of ML systems beyond the traditional training set/test set paradigm.
Not checkable · 24 Sep 2020 |
|
| Not checkable |
Not checkable |
18 Sep 2020 |
But my endorsement is in defense of democracy & opposition to authoritarianism kleptocracy, corruption, divisiveness.
Not checkable · 18 Sep 2020 |
|
| Not checkable |
Not checkable |
18 Sep 2020 |
Immigration & Science-based policies are paramount.
Not checkable · 18 Sep 2020 |
|
| Not checkable |
Not checkable |
17 Sep 2020 |
Advocating for whole mouse brain connectomics, with many distinguished colleagues.
Not checkable · 17 Sep 2020 |
|
| Not checkable |
Not checkable |
16 Sep 2020 |
Very cool work from UCL+FAIR-London on neural architectures for reasoning that learn to generate and select rules.
Not checkable · 16 Sep 2020 |
|
| Not checkable |
Not checkable |
2 Sep 2020 |
Awesome work from FAIR on "protein embedding" using pre-trained transformer architectures.
Not checkable · 2 Sep 2020 |
|
| Not checkable |
Not checkable |
24 Aug 2020 |
Awesome work from FAIR.
Not checkable · 24 Aug 2020 |
|
| Not checkable |
Not checkable |
22 Aug 2020 |
Republicans who were critics of Trump before becoming critics of his critics are hypocrites.
Not checkable · 22 Aug 2020 |
|
| Not checkable |
Not checkable |
13 Aug 2020 |
Another advance for normalizing flows in high-energy physics simulations.
Not checkable · 13 Aug 2020 |
|
| Not checkable |
Not checkable |
9 Aug 2020 |
Very nice, intuitive and visual explanation for the "double descent" phenomenon that produces a peak in test error when the model capacity…
Not checkable · 9 Aug 2020 |
|
| Not checkable |
Not checkable |
7 Aug 2020 |
I think "jiga" is somewhat closer to the Greek pronounciation of γίγας
Not checkable · 7 Aug 2020 |
|
| Not checkable |
Not checkable |
28 Jul 2020 |
Oh, and meters and kg not feet and pounds.
Not checkable · 28 Jul 2020 |
|
| Not checkable |
Not checkable |
28 Jul 2020 |
Yeah, we also say retropropagation not backpropagation, "convolution à trous" not "dilated convolution", "Dirac distribution" not "delta…
Not checkable · 28 Jul 2020 |
|
| Not checkable |
Not checkable |
25 Jul 2020 |
I have an overly complicated joke about convolutions but it's simpler in the Fourier domain.
Not checkable · 25 Jul 2020 |
|
| Not checkable |
Not checkable |
29 Jun 2019 |
When pigs fly.
Not checkable · 29 Jun 2019 |
|