| Not checkable |
Not checkable |
27 Jun 2019 |
Excruciating details on my life as a scientist larva, which make it painfully obvious that I'm nowhere near the "brightest minds in…
Not checkable · 27 Jun 2019 |
|
| Not checkable |
Not checkable |
26 Jun 2019 |
Congrats Noam!
Not checkable · 26 Jun 2019 |
|
| Not checkable |
Not checkable |
26 Jun 2019 |
David Brooks' writing is so much better when he doesn't write about politics.
Not checkable · 26 Jun 2019 |
|
| Not checkable |
Not checkable |
25 Jun 2019 |
Welcome Lerrel!
Not checkable · 25 Jun 2019 |
|
| Not checkable |
Not checkable |
23 Jun 2019 |
This report on Trump's concentration camps for migrant children is absolutely horrifying.
Not checkable · 23 Jun 2019 |
|
| Not checkable |
Not checkable |
22 Jun 2019 |
but you can't change the laws of physics.
Not checkable · 22 Jun 2019 |
|
| Not checkable |
Not checkable |
21 Jun 2019 |
Great interview with Peter Shor.
Not checkable · 21 Jun 2019 |
|
| Not checkable |
Not checkable |
17 Jun 2019 |
self-supervised learning is the cake,
Not checkable · 17 Jun 2019 |
|
| Not checkable |
Not checkable |
15 Jun 2019 |
Interesting NBER paper on the history of industry investment in basic research.
Not checkable · 15 Jun 2019 |
|
| Not checkable |
Not checkable |
13 Jun 2019 |
They started by putting children in concentration camps.
Not checkable · 13 Jun 2019 |
|
| Not checkable |
Not checkable |
13 Jun 2019 |
Brought to you from FAIR.
Not checkable · 13 Jun 2019 |
|
| Not checkable |
Not checkable |
10 Jun 2019 |
I never understood the vinyl fetishism that some music amateurs possess.
Not checkable · 10 Jun 2019 |
|
| Not checkable |
Not checkable |
7 Jun 2019 |
Nice new work from FAIR on assessing bias in datasets.
Not checkable · 7 Jun 2019 |
|
| Not checkable |
Not checkable |
6 Jun 2019 |
This has got to be the most hilariously ridiculous thing I've seen in a while.
Not checkable · 6 Jun 2019 |
|
| Not checkable |
Not checkable |
6 Jun 2019 |
Nice collection of PyTorch (and some TF) Jupyter notebooks for everything deep learning by Sabastian Raschka.
Not checkable · 6 Jun 2019 |
|
| Not checkable |
Not checkable |
3 Jun 2019 |
not as effective as you might think.
Not checkable · 3 Jun 2019 |
|
| Not checkable |
Not checkable |
3 Jun 2019 |
Which may explain why trolls and propagandists haven't used it much
Not checkable · 3 Jun 2019 |
|
| Not checkable |
Not checkable |
3 Jun 2019 |
more easily detectable as you might think.
Not checkable · 3 Jun 2019 |
|
| Not checkable |
Not checkable |
1 Jun 2019 |
Pretty cool.
Not checkable · 1 Jun 2019 |
|
| Not checkable |
Not checkable |
31 May 2019 |
Thank you to the French Consulate in NYC for hosting this event.
Not checkable · 31 May 2019 |
|
| Not checkable |
Not checkable |
2019 |
indicates the number of parameters [22].
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The important lesson is that Hardware capabilities and software tools both motivate and limit the type of ideas that AI researchers will…
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The ability to predict what is going to happen in the world is what allows us to learn to drive without causing accidents
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Price and power consumption are relatively secondary to performance and flexibility.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
This model can be used to plan or to train
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Obviously, our current learning paradigms are missing a key ingredient.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Early neural network chips from Bell Labs.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Using FP-32 is necessary because one must be sure that when an experiment fails, it is not because of a lack of numerical accuracy.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Many problems are difficult to represent with fixed-size tensors or variable-length sequences of tensors, but are better represented by…
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
In effect, dynamic DL systems can be seen as the execution trace of a program, with conditionals and loops that are input-dependent.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Multi-Scale DenseNet with conditional computation
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
There are five use cases with different hardware requirements
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The best architectures are those that can be saturated with the smallest batch of samples.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The answer is a resounding yes.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The number of trials required is large because the feedback from the environment is information-poor.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
This kind of model can be used to predict multiple scenarios of how surrounding cars are going to move, and to plan a driving policy…
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Video prediction is a subject of wide interest because it may allow future systems, such as robots or self-driving cars, to predict what is…
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
new ideas seem limited by the available hardware (and software)
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The word “architecture” in the context of DL designates the graph of functional modules, not the architecture of the underlying hardware.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
system-level performance is considerably more important than raw speed
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Good results are essential but not sufficient.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The main difficulty is that predicting the future of a video is not achievable exactly because the world is not entirely predictable.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
What has accounted for the success of GPGPU for DL is their wide availability, generality, programmability, and well-supported software…
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
One hypothesis is that this missing ingredient is self-supervised learning.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
But there is no algorithmic reason to batch multiple samples.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
non-programmability was a major limitation
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The use of predictive models that not only predict the evolution of the environment, but also predict the consequences of actions, is key…
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Learning Applied to Document Recognition
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The key advantage of DL is that it alleviates the need to hand-design a feature extractor, as would be required with traditional machine…
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
If the energyminimizing inference procedure is gradient-based, inference hardware will need to support back-propagation.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The tools at our disposal fashion our thoughts more than we care to admit.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Not all layers are represented [37].
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The ideal architecture is a specialized DL-inference accelerator sitting in a standard data-center
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The idea of self-supervised learning is to train a machine to predict any subset of its input from other subsets
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
It is very unclear whether analog implementations provide any power dissipation advantages over digital
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The scores are normalized to sum to one.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The history of neural networks is inextricably linked with hardware.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
An example of Convolutional Network architecture for
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The phrase “differentiable programming” designates the process of writing a program with calls to parameterized functions that…
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
For a review/tutorial on energy-based models and structured prediction on top of neural nets (or other models) see this paper
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
It allows us to train a discriminator as an unsupervised “density estimator”, i.e. a contrast function that gives us a low value for data…
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
A particular instance of energy-based models is factor graphs (non-probabilistic graphical models).
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
To enable deep learning systems to reason, we need to modify them so that they don’t produce a single output
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
But perhaps more interestingly, the generator can be seen as parameterizing the complicated surface of real data: give it a vector Z, and…
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
There are papers where people do amazing things with this, like generating pictures of bedrooms, doing arithmetic on faces in the Z vector…
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
This discriminator has to develop a good internal representation of the data to solve this problem properly.
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
This is what energy-based models are designed to do: give you a score for each possible configuration of the variables to be inferred.
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
The most important one, in my opinion, is adversarial training (also called GAN for Generative Adversarial Networks).
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
The Deep Learning textbook by Goodfellow, Bengio and Courville
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
an overview paper in Nature by myself, Yoshua Bengio and Geoff Hinton with lots of pointers to the literature
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
the Coursera course on neural nets by Geoff Hinton (starting to be a bit dated).
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
Combining learning systems with factor graphs is known as “structured prediction” in machine learning.
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
It can then be used as a feature extractor for a classifier, for example.
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
This, and the variations that are now being proposed is the most interesting idea in the last 10 years in ML, in my opinion.
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
Jan 2016 |
How I learned to stop worrying and love unsupervised learning
Not checkable · Jan 2016 |
|
| Not checkable |
Not checkable |
Jan 2016 |
the cherry on the cake
Not checkable · Jan 2016 |
|
| Not checkable |
Not checkable |
27 May 2015 |
recurrent nets have shone light on sequential data such as text and speech.
Not checkable · 27 May 2015 |
|
| Not checkable |
Not checkable |
27 May 2015 |
The authors would like to thank the Natural Sciences and Engineering Research Council of Canada, the Canadian Institute For Advanced…
Not checkable · 27 May 2015 |
|
| Not checkable |
Not checkable |
27 May 2015 |
Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple…
Not checkable · 27 May 2015 |
|
| Not checkable |
Not checkable |
27 May 2015 |
Deep learning discovers intricate structure in large data sets
Not checkable · 27 May 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
IBM built the wrong thing.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
It’s a very bad way to approach an engineering problem.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
The problem with symbols is that a symbol is a meaningless string of bits.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
I’m not pompous, I think, and Facebook is a very non-pompous company.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
It is not the learning that is shallow or deep, but the architecture that is being trained.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
Our models do not aspire to be models of the brain, and we don’t make claims of neural relevance.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
You could think of Deep Learning as the building of learning machines, say pattern recognition systems or whatever, by assembling lots of…
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
Interacting with PhD students and young researchers is a very good remedy against hubris.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
But designing a feature extractor “by hand” is very difficult and time consuming.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
There is so much fun I can have with the work. But sometimes I need to build things with my hands.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
But there are things that are worth worrying about today, and there are things that are so far out that we can write science fiction about…
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
So in my opinion—and I used to be a chip designer—you should build a chip only when you’re pretty damn sure it can do something useful.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
I have always been enamored of the idea of being able to train an entire system from end to end.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
I don’t think moral questions should be left to scientists alone!
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
Actually, I think the basics of machine learning are quite simple to understand.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
describing it like the brain gives a bit of the aura of magic to it, which is dangerous.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
I think it’s nuts.
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
By training the system to predict what the world is going to be like a second, a minute, an hour, or a day from now, you can train it to…
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
That’s why we don’t like hype: it is made by people who are either dishonest or self-deluded, and makes the life of serious and honest…
Not checkable · 18 Feb 2015 |
|
| Not checkable |
Not checkable |
18 Feb 2015 |
The problem is that it’s not going to be able to recognize objects to any degree of accuracy, unless you make it so gigantically big that…
Not checkable · 18 Feb 2015 |
|