| Not checkable |
Not checkable |
3 Jun 2019 |
Which may explain why trolls and propagandists haven't used it much
Not checkable · 3 Jun 2019 |
|
| Not checkable |
Not checkable |
3 Jun 2019 |
more easily detectable as you might think.
Not checkable · 3 Jun 2019 |
|
| Mostly true |
Factual |
3 Jun 2019 |
The Beijing Academy of Artificial Intelligence's published AI ethics guidelines mention the protection of individual privacy.
Factual · 3 Jun 2019 |
AI policy and governance |
| Not researched |
Factual |
2 Jun 2019 |
The Belgian newspaper L'Écho published an article on Facebook's robotics research efforts.
Factual · 2 Jun 2019 |
Technology |
| Not checkable |
Not checkable |
1 Jun 2019 |
Pretty cool.
Not checkable · 1 Jun 2019 |
|
| Not checkable |
Not checkable |
31 May 2019 |
Thank you to the French Consulate in NYC for hosting this event.
Not checkable · 31 May 2019 |
|
| Not researched |
Factual |
2019 |
Convolution operations can be defined on irregular graphs as diagonal operators in the eigenspace of the graph Laplacian.
Factual · 2019 |
AI capabilities |
| Not checkable |
Not checkable |
2019 |
indicates the number of parameters [22].
Not checkable · 2019 |
|
| Resolved true |
Factual |
2019 |
In computer vision, the trend is to apply ConvNets to an entire image to detect, segment and recognize objects of any size at any location.
Factual · 2019 |
AI capabilities |
| Not checkable |
Not checkable |
2019 |
The important lesson is that Hardware capabilities and software tools both motivate and limit the type of ideas that AI researchers will…
Not checkable · 2019 |
|
| Unresolvable |
Factual |
2019 |
Current model-free RL methods would likely need millions of hours of practice, with numerous accidents, for an autonomous car to learn to drive.
Factual · 2019 |
AI capabilities |
| Unresolvable |
Factual |
2019 |
Training a video predictor with a least-square criterion yields blurry frames that average all possible futures.
Factual · 2019 |
AI capabilities |
| Not checkable |
Not checkable |
2019 |
The ability to predict what is going to happen in the world is what allows us to learn to drive without causing accidents
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Price and power consumption are relatively secondary to performance and flexibility.
Not checkable · 2019 |
|
| Not researched |
Factual |
2019 |
Modern AI is powered by deep learning, whose origins go back to early experiments with electronic neural nets in the 1950s.
Factual · 2019 |
AI capabilities |
| Not checkable |
Not checkable |
2019 |
This model can be used to plan or to train
Not checkable · 2019 |
|
| Not researched |
Factual |
2019 |
Most of today's deep learning systems simply produce an output given an input.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
Python is very slow and memory hungry.
Factual · 2019 |
Technology |
| Not researched |
Factual |
2019 |
Facebook performs 3×10^14 predictions per day, some of which are run on mobile devices.
Factual · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
Yann LeCun said that after joining in late 1988, he developed the first convolutional neural net and obtained excellent results on handwritten character recognition tasks (zip codes).
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
Around 2003, Hinton, Bengio and LeCun, with CIFAR funding, started research projects, workshops and summer schools with the goal of reviving ML community interest in neural networks.
Factual · 2019 |
Field building and careers |
| Not checkable |
Not checkable |
2019 |
Obviously, our current learning paradigms are missing a key ingredient.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Early neural network chips from Bell Labs.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Using FP-32 is necessary because one must be sure that when an experiment fails, it is not because of a lack of numerical accuracy.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Many problems are difficult to represent with fixed-size tensors or variable-length sequences of tensors, but are better represented by…
Not checkable · 2019 |
|
| Not researched |
Factual |
2019 |
DL architecture research is evolving toward more sophisticated and dynamic network architectures, and inputs and internal states that are graphs annotated with numerical objects.
Factual · 2019 |
AI capabilities |
| Not checkable |
Not checkable |
2019 |
In effect, dynamic DL systems can be seen as the execution trace of a program, with conditionals and loops that are input-dependent.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Multi-Scale DenseNet with conditional computation
Not checkable · 2019 |
|
| Not researched |
Factual |
2019 |
On average, neurons in the brain are at 2% of their maximum activation.
Factual · 2019 |
Science and research methods |
| Not researched |
Factual |
2019 |
In the early days, training a simple ConvNet for handwritten character recognition could take weeks.
Factual · 2019 |
AI compute, hardware and data |
| Unresolvable |
Prediction |
2019 |
The demands on DL-specific hardware will undoubtedly increase.
Prediction · 2019 |
AI compute, hardware and data |
| Not checkable |
Not checkable |
2019 |
There are five use cases with different hardware requirements
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The best architectures are those that can be saturated with the smallest batch of samples.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The answer is a resounding yes.
Not checkable · 2019 |
|
| Not researched |
Factual |
2019 |
Large-scale embedding methods that learn vectors per object and subject are particularly efficient for recommender systems.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
The performance of spiking neuron circuits seems considerably inferior to that of traditional digital architectures for realistic ConvNet-type networks.
Factual · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
RL's sample inefficiency is not a problem in easy-to-simulate, fully observable, discrete-state environments such as go or chess.
Factual · 2019 |
AI capabilities |
| Not checkable |
Not checkable |
2019 |
The number of trials required is large because the feedback from the environment is information-poor.
Not checkable · 2019 |
|
| Unresolvable |
Prediction |
2019 |
Chances are that the bulk of computation in future deep learning systems will still consist primarily of convolutions.
Prediction · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
Pooling makes a ConvNet representation robust to small shifts in the location of distinctive features.
Factual · 2019 |
AI capabilities |
| Unresolvable |
Prediction |
2019 |
Programmable resistor technology may become sufficiently compact, reliable, durable and configurable for deep learning applications.
Prediction · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
By the late 1990s, between 10% and 20% of all bank checks in the US were automatically read by the ConvNet-based system developed by LeCun's team.
Factual · 2019 |
Economics and markets |
| Not checkable |
Not checkable |
2019 |
This kind of model can be used to predict multiple scenarios of how surrounding cars are going to move, and to plan a driving policy…
Not checkable · 2019 |
|
| Not researched |
Factual |
2019 |
Almost all practical applications of deep learning use supervised learning, with a tiny minority using reinforcement learning.
Factual · 2019 |
AI capabilities |
| Not checkable |
Not checkable |
2019 |
Video prediction is a subject of wide interest because it may allow future systems, such as robots or self-driving cars, to predict what is…
Not checkable · 2019 |
|
| Not researched |
Factual |
2019 |
Deep learning systems tend to work better the larger they are made.
Factual · 2019 |
AI capabilities |
| Not checkable |
Not checkable |
2019 |
new ideas seem limited by the available hardware (and software)
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The word “architecture” in the context of DL designates the graph of functional modules, not the architecture of the underlying hardware.
Not checkable · 2019 |
|
| Not researched |
Factual |
2019 |
The upper layers of ConvNets are typically less compute bound but more memory bound.
Factual · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
Variations of GANs have produced stunning results in image generation.
Factual · 2019 |
AI capabilities |
| Not checkable |
Not checkable |
2019 |
system-level performance is considerably more important than raw speed
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
Good results are essential but not sufficient.
Not checkable · 2019 |
|
| Unresolvable |
Prediction |
2019 |
Augmented/virtual reality and telepresence applications will require extremely low-power ASICs for DL inference.
Prediction · 2019 |
AI compute, hardware and data |
| Not checkable |
Not checkable |
2019 |
The main difficulty is that predicting the future of a video is not achievable exactly because the world is not entirely predictable.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
What has accounted for the success of GPGPU for DL is their wide availability, generality, programmability, and well-supported software…
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
One hypothesis is that this missing ingredient is self-supervised learning.
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
But there is no algorithmic reason to batch multiple samples.
Not checkable · 2019 |
|
| Not researched |
Factual |
2019 |
In practical ConvNet applications, much of the computation is spent on convolutions in the lower layers.
Factual · 2019 |
AI compute, hardware and data |
| Not checkable |
Not checkable |
2019 |
non-programmability was a major limitation
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The use of predictive models that not only predict the evolution of the environment, but also predict the consequences of actions, is key…
Not checkable · 2019 |
|
| Mostly true |
Factual |
2019 |
The ConvNet architecture was loosely inspired by the ventral pathway in the visual cortex.
Factual · 2019 |
AI capabilities |
| Mostly true |
Factual |
2019 |
ReLU nonlinearities and dropout regularization enabled purely supervised backprop training of very deep networks with a dozen layers.
Factual · 2019 |
AI capabilities |
| Not checkable |
Not checkable |
2019 |
Learning Applied to Document Recognition
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The key advantage of DL is that it alleviates the need to hand-design a feature extractor, as would be required with traditional machine…
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
If the energyminimizing inference procedure is gradient-based, inference hardware will need to support back-propagation.
Not checkable · 2019 |
|
| Unresolvable |
Factual |
2019 |
Dataflow stream processors have become a popular design for ConvNet accelerators, such as the Eyeriss project.
Factual · 2019 |
AI compute, hardware and data |
| Unresolvable |
Prediction |
2019 |
The ultimate solution to DL power constraints may be exploiting sparse activations, perhaps with event-based computation.
Prediction · 2019 |
AI compute, hardware and data |
| Not checkable |
Not checkable |
2019 |
The tools at our disposal fashion our thoughts more than we care to admit.
Not checkable · 2019 |
|
| Not researched |
Factual |
2019 |
The original 1957 Perceptron was a dedicated analog computer whose weights were implemented with motorized potentiometers.
Factual · 2019 |
AI compute, hardware and data |
| Not researched |
Citation |
2019 |
Reference [35] shows that an 8-bit logarithmic number system with a Kulisch accumulator significantly reduces power consumption with negligible accuracy loss for ResNet-50 on ImageNet.
Citation · 2019 |
AI compute, hardware and data |
| Unresolvable |
Prediction |
2019 |
What the AI hardware and software communities produce over the next few years will shape AI research for the coming decades.
Prediction · 2019 |
AI compute, hardware and data |
| Not checkable |
Not checkable |
2019 |
Not all layers are represented [37].
Not checkable · 2019 |
|
| Unresolvable |
Prediction |
2019 |
Self-supervised latent-variable predictive models will be the centerpiece of intelligent systems based on model-predictive control and model-based reinforcement learning, e.g. for robotic manipulation and autonomous driving.
Prediction · 2019 |
AI capabilities |
| Not checkable |
Not checkable |
2019 |
The ideal architecture is a specialized DL-inference accelerator sitting in a standard data-center
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The idea of self-supervised learning is to train a machine to predict any subset of its input from other subsets
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
It is very unclear whether analog implementations provide any power dissipation advantages over digital
Not checkable · 2019 |
|
| Not researched |
Factual |
2019 |
The amount of raw video data available to train self-supervised systems is essentially unlimited.
Factual · 2019 |
AI compute, hardware and data |
| Mostly false |
Factual |
2019 |
Users upload 2 to 3 billion photos on Facebook every day.
Factual · 2019 |
AI compute, hardware and data |
| Not checkable |
Not checkable |
2019 |
The scores are normalized to sum to one.
Not checkable · 2019 |
|
| Not researched |
Factual |
2019 |
The SN system inaugurated the idea of neural networks as computational graphs of standard parameterized modules with automatic differentiation.
Factual · 2019 |
AI capabilities |
| Mostly true |
Prediction |
2019 |
Usage of graph neural networks will increase for a wide variety of applications likely to violate the assumptions of current deep learning hardware.
Prediction · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
Léon Bottou and Yann LeCun started developing a neural-network simulator with a home-grown Lisp-dialect front end in 1987.
Factual · 2019 |
AI capabilities |
| Not checkable |
Not checkable |
2019 |
The history of neural networks is inextricably linked with hardware.
Not checkable · 2019 |
|
| Unresolvable |
Prediction |
2019 |
New high-performance hardware will be required to enable progress towards deep learning systems trained to acquire vast amounts of background knowledge and a form of common sense.
Prediction · 2019 |
AI compute, hardware and data |
| Not checkable |
Not checkable |
2019 |
An example of Convolutional Network architecture for
Not checkable · 2019 |
|
| Not checkable |
Not checkable |
2019 |
The phrase “differentiable programming” designates the process of writing a program with calls to parameterized functions that…
Not checkable · 2019 |
|
| Mostly true |
Prediction |
2019 |
As DL is used for high-level cognitive tasks, memory modules will become commonplace and very large, requiring hardware support.
Prediction · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
Sparse activation is one of the features that makes the brain power-efficient.
Factual · 2019 |
AI compute, hardware and data |
| Not checkable |
Not checkable |
1 Aug 2016 |
For a review/tutorial on energy-based models and structured prediction on top of neural nets (or other models) see this paper
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
It allows us to train a discriminator as an unsupervised “density estimator”, i.e. a contrast function that gives us a low value for data…
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
A particular instance of energy-based models is factor graphs (non-probabilistic graphical models).
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
To enable deep learning systems to reason, we need to modify them so that they don’t produce a single output
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
But perhaps more interestingly, the generator can be seen as parameterizing the complicated surface of real data: give it a vector Z, and…
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
There are papers where people do amazing things with this, like generating pictures of bedrooms, doing arithmetic on faces in the Z vector…
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
This discriminator has to develop a good internal representation of the data to solve this problem properly.
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
This is what energy-based models are designed to do: give you a score for each possible configuration of the variables to be inferred.
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
The most important one, in my opinion, is adversarial training (also called GAN for Generative Adversarial Networks).
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
The Deep Learning textbook by Goodfellow, Bengio and Courville
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
an overview paper in Nature by myself, Yoshua Bengio and Geoff Hinton with lots of pointers to the literature
Not checkable · 1 Aug 2016 |
|
| Not checkable |
Not checkable |
1 Aug 2016 |
the Coursera course on neural nets by Geoff Hinton (starting to be a bit dated).
Not checkable · 1 Aug 2016 |
|