| Not researched |
Factual |
Jun 2023 |
SSL works very well for text, but for images, models trained to make a single prediction predict the average of all possible predictions.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Citation |
Jun 2023 |
The method for maximizing information content of learned representations is called variance-invariance-covariance regularization (VICReg), per reference [58].
Citation · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
Computing a partition function is very challenging, so probabilistic models in continuous high-dimensional domains are intractable.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
Some regularized autoencoders learn interpretable features that mirror existing hand-crafted data compression approaches.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
A major technique for detecting blacklisted content is nearest-neighbor search against that content.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
Current self-driving models need thousands of footage to learn, while humans can learn to drive in about twenty hours.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
2021 |
Contrastive learning with positive and negative pairs does not work when the representation is high-dimensional, because there are too many ways for two things to be different.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
In MIT experiments where a baby ferret's optic nerve is connected to the auditory cortex, the auditory cortex becomes essentially a visual cortex.
Factual · 2021 |
Science and research methods |
| Not researched |
Factual |
2021 |
In the first months of life, babies learn how the world works mostly by observation because they can hardly act in the world.
Factual · 2021 |
Philosophy, ethics and society |
| Not researched |
Factual |
2021 |
Across AI sub-fields there has been a continuous evolution from less hand-crafting to more learning.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
People have used data augmentation on MNIST digits since the 1990s.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
Geoffrey Hinton proposed, in the early 1990s with his student Sue Becker, the idea of maximizing mutual information between the outputs of two systems.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
After self-supervised representation learning, recognizing handwritten digits on top of it requires only a few labeled samples.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
The difference between human and chimp genomes involved in human intelligence is less than 1% of the genome.
Factual · 2021 |
Science and research methods |
| Not researched |
Factual |
2021 |
The system under discussion worked, but they ended up not using it.
Factual · 2021 |
Technology |
| Not researched |
Factual |
2021 |
LeCun used to call the two-identical-copies network architecture a Siamese network.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
With a differentiable world model such as a neural net, one can do gradient-based model predictive control by backpropagating through it.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
Orangutans are solitary animals that do not seek the company of others.
Factual · 2021 |
Other |
| Not researched |
Factual |
2021 |
Babies learn an enormous amount of background knowledge about the world that may be the basis of common sense.
Factual · 2021 |
Philosophy, ethics and society |
| Not researched |
Factual |
2021 |
Self-supervised learning offers far more training signal than supervised or reinforcement learning.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
Generally, the model is mostly deterministic and mostly built by hand, though sometimes a few parameters have to be identified through what is called systems identification.
Factual · 2021 |
Technology |
| Not researched |
Factual |
2021 |
Barlow Twins is a method for maximizing the information content of a vector.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
The first vision systems in animals evolved mainly for localization, with very little recognition.
Factual · 2021 |
Science and research methods |
| Not researched |
Factual |
2021 |
Representations from self-supervised methods that eliminate augmentation distortions cannot be used for object detection and localization because that information is lost.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
Yann LeCun proposes self-supervised learning as the way to get machines to learn to represent the world.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
A sea change in favor of convolutional nets occurred when datasets became bigger and GPUs became available.
Factual · 2021 |
AI compute, hardware and data |
| Not researched |
Factual |
2021 |
Until very recently there were no working methods for training image representations by masking parts of images and reconstructing them.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
Oriented edge detectors can be learned within minutes of opening one's eyes.
Factual · 2021 |
Science and research methods |
| Not researched |
Factual |
2021 |
LeCun said he believes no text in the world explains the physical situation he was describing (an object that is "sitting on" something else).
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
In the early 1990s LeCun and Bell Labs colleagues including Jane Bromley proposed using negative example pairs, now called contrastive learning.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
LeCun revived Hinton's idea together with his postdocs at FAIR, producing a method called Barlow Twins.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
Model predictive control in classical optimal control has existed since the early 1960s.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
The distinguishing aspects of human intelligence appeared only in the last million years.
Factual · 2021 |
Science and research methods |
| Not researched |
Factual |
2021 |
Without specific provisions, a Siamese network trained on similar pairs ends up ignoring its input and producing a constant output vector.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
Zeroth-order, gradient-free black-box optimization is much less efficient than gradient-based optimization.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
Self-supervised pre-training has been less successful in images than in language, is making progress, and relies on manual data augmentation.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
LeCun said he thinks the entire human prefrontal cortex is basically the engine for a world model.
Factual · 2021 |
Science and research methods |
| Not researched |
Factual |
2021 |
In supervised learning the machine is given a few bits of feedback per sample.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
A paper coming out of the FAIR group at Menlo Park shows a masking-based method that works very well, without data augmentation.
Factual · 2021 |
Their own work or organisation |
| Not researched |
Factual |
2021 |
Lions attack bigger animals than cheetahs do.
Factual · 2021 |
Other |
| Not researched |
Factual |
2021 |
LeCun said he thinks oriented edge detectors in the visual cortex area V1 are learned rather than hardwired.
Factual · 2021 |
Science and research methods |
| Not researched |
Factual |
2021 |
Every modern natural language processing system is pre-trained in a self-supervised manner by filling in blanks.
Factual · 2021 |
AI capabilities |
| Not researched |
Factual |
2021 |
Signals from the retina can easily train edge detectors, so V1 edge detectors may be learned before birth.
Factual · 2021 |
Science and research methods |
| Not researched |
Factual |
2021 |
Octopuses never know their parents.
Factual · 2021 |
Other |
| Not researched |
Factual |
26 Sep 2020 |
Siamese networks trained to tell whether two states of a physical system lie on the same trajectory appear to discover symmetries and conservation laws.
Factual · 26 Sep 2020 |
Science and research methods |
| Not researched |
Factual |
18 Sep 2020 |
Yann LeCun is one of 24 American Turing Award laureates who endorsed the Biden/Harris ticket.
Factual · 18 Sep 2020 |
AI policy and governance |
| Not researched |
Factual |
10 Sep 2020 |
The Legendre transform is like the Fourier transform with the (min,+) semi-ring instead of the (+,×) algebra.
Factual · 10 Sep 2020 |
Science and research methods |
| Not researched |
Factual |
4 Sep 2020 |
Hydra 1.0 has been released.
Factual · 4 Sep 2020 |
Technology |
| Not researched |
Factual |
4 Sep 2020 |
The 2020 ML Reproducibility Challenge has been announced.
Factual · 4 Sep 2020 |
Science and research methods |
| Not researched |
Factual |
3 Sep 2020 |
Sebastian Seung was using ConvNets for connectomics long before deep learning became popular.
Factual · 3 Sep 2020 |
Science and research methods |
| Not researched |
Citation |
18 Aug 2020 |
A new FastMRI paper shows that patient time in an MRI machine can be divided by 4 with no degradation in image quality.
Citation · 18 Aug 2020 |
Biosecurity |
| Not researched |
Factual |
14 Aug 2020 |
Facebook AI Research (FAIR) has produced trainable Minecraft bots.
Factual · 14 Aug 2020 |
AI capabilities |
| Not researched |
Factual |
30 Jun 2019 |
Facebook announced additional measures against hate speech and discrimination on its platform.
Factual · 30 Jun 2019 |
Philosophy, ethics and society |
| Not researched |
Factual |
26 Jun 2019 |
Torchvision 0.3 was released by 2019-06-26.
Factual · 26 Jun 2019 |
Technology |
| Not researched |
Factual |
24 Jun 2019 |
Yann LeCun stated that he and Geoff Hinton delivered the Turing Lecture earlier on the day of his 2019-06-24 post, at the ACM Federated Computing Research Conference in Phoenix.
Factual · 24 Jun 2019 |
Their own work or organisation |
| Not researched |
Factual |
23 Jun 2019 |
The ResNeXt-101 32x8d model pre-trained on Instagram hashtags and fine-tuned on ImageNet, with 85.4% top-1 accuracy, is now available.
Factual · 23 Jun 2019 |
Technology |
| Not researched |
Factual |
19 Jun 2019 |
FAIR and NYU Langone Medical Center developed a method for faster MRI through active acquisition.
Factual · 19 Jun 2019 |
Science and research methods |
| Not researched |
Factual |
16 Jun 2019 |
Yann LeCun attended the ACM Award banquet in San Francisco on 2019-06-15 with Yoshua Bengio, Susan Dumais, Eric Horvitz and Geoffrey Hinton.
Factual · 16 Jun 2019 |
Their own work or organisation |
| Not researched |
Factual |
13 Jun 2019 |
FAIR produced work applying Energy-Based Models to detecting machine-generated versus human-written text.
Factual · 13 Jun 2019 |
AI capabilities |
| Not researched |
Factual |
13 Jun 2019 |
Apparently, GANs are used to create fake LinkedIn profile pictures for international industrial espionage.
Factual · 13 Jun 2019 |
AI safety and alignment |
| Not researched |
Factual |
2 Jun 2019 |
The Belgian newspaper L'Écho published an article on Facebook's robotics research efforts.
Factual · 2 Jun 2019 |
Technology |
| Not researched |
Factual |
2019 |
Convolution operations can be defined on irregular graphs as diagonal operators in the eigenspace of the graph Laplacian.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
Modern AI is powered by deep learning, whose origins go back to early experiments with electronic neural nets in the 1950s.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
Most of today's deep learning systems simply produce an output given an input.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
Python is very slow and memory hungry.
Factual · 2019 |
Technology |
| Not researched |
Factual |
2019 |
Facebook performs 3×10^14 predictions per day, some of which are run on mobile devices.
Factual · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
Yann LeCun said that after joining in late 1988, he developed the first convolutional neural net and obtained excellent results on handwritten character recognition tasks (zip codes).
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
Around 2003, Hinton, Bengio and LeCun, with CIFAR funding, started research projects, workshops and summer schools with the goal of reviving ML community interest in neural networks.
Factual · 2019 |
Field building and careers |
| Not researched |
Factual |
2019 |
DL architecture research is evolving toward more sophisticated and dynamic network architectures, and inputs and internal states that are graphs annotated with numerical objects.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
On average, neurons in the brain are at 2% of their maximum activation.
Factual · 2019 |
Science and research methods |
| Not researched |
Factual |
2019 |
In the early days, training a simple ConvNet for handwritten character recognition could take weeks.
Factual · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
Large-scale embedding methods that learn vectors per object and subject are particularly efficient for recommender systems.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
The performance of spiking neuron circuits seems considerably inferior to that of traditional digital architectures for realistic ConvNet-type networks.
Factual · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
RL's sample inefficiency is not a problem in easy-to-simulate, fully observable, discrete-state environments such as go or chess.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
Pooling makes a ConvNet representation robust to small shifts in the location of distinctive features.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
By the late 1990s, between 10% and 20% of all bank checks in the US were automatically read by the ConvNet-based system developed by LeCun's team.
Factual · 2019 |
Economics and markets |
| Not researched |
Factual |
2019 |
Almost all practical applications of deep learning use supervised learning, with a tiny minority using reinforcement learning.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
Deep learning systems tend to work better the larger they are made.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
The upper layers of ConvNets are typically less compute bound but more memory bound.
Factual · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
Variations of GANs have produced stunning results in image generation.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
In practical ConvNet applications, much of the computation is spent on convolutions in the lower layers.
Factual · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
The original 1957 Perceptron was a dedicated analog computer whose weights were implemented with motorized potentiometers.
Factual · 2019 |
AI compute, hardware and data |
| Not researched |
Citation |
2019 |
Reference [35] shows that an 8-bit logarithmic number system with a Kulisch accumulator significantly reduces power consumption with negligible accuracy loss for ResNet-50 on ImageNet.
Citation · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
The amount of raw video data available to train self-supervised systems is essentially unlimited.
Factual · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
2019 |
The SN system inaugurated the idea of neural networks as computational graphs of standard parameterized modules with automatic differentiation.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
Léon Bottou and Yann LeCun started developing a neural-network simulator with a home-grown Lisp-dialect front end in 1987.
Factual · 2019 |
AI capabilities |
| Not researched |
Factual |
2019 |
Sparse activation is one of the features that makes the brain power-efficient.
Factual · 2019 |
AI compute, hardware and data |
| Not researched |
Factual |
18 Feb 2015 |
Unsupervised learning techniques that actually work now exist.
Factual · 18 Feb 2015 |
AI capabilities |
| Not researched |
Factual |
18 Feb 2015 |
With hundreds of millions of faces in the gallery, face recognition accuracy is nowhere near 97.25 percent.
Factual · 18 Feb 2015 |
AI capabilities |
| Not researched |
Factual |
18 Feb 2015 |
Facebook's motto is to connect people.
Factual · 18 Feb 2015 |
Other |
| Not researched |
Factual |
18 Feb 2015 |
Earlier shallow learning systems were limited in the complexity of the functions they could compute.
Factual · 18 Feb 2015 |
AI capabilities |
| Not researched |
Factual |
18 Feb 2015 |
The type of learning used in actual Deep Learning systems is very restricted, with supervised learning being what works in practice.
Factual · 18 Feb 2015 |
AI capabilities |
| Not researched |
Factual |
18 Feb 2015 |
AI has gone through a number of AI winters because people claimed things they could not deliver.
Factual · 18 Feb 2015 |
AI timelines and forecasting |
| Not researched |
Factual |
18 Feb 2015 |
Computer facial recognition on real Internet pictures would be pretty close to human performance.
Factual · 18 Feb 2015 |
AI capabilities |
| Not researched |
Factual |
18 Feb 2015 |
IBM's rat-scale brain simulation on a supercomputer was just a random network of neurons that did nothing useful.
Factual · 18 Feb 2015 |
AI capabilities |
| Not researched |
Factual |
18 Feb 2015 |
Humans and animals do not learn through supervised labelling of examples the way Deep Learning systems do.
Factual · 18 Feb 2015 |
Science and research methods |
| Not researched |
Factual |
18 Feb 2015 |
Facebook has to automatically select 100 to 150 items that users want or need to see.
Factual · 18 Feb 2015 |
AI capabilities |
| Not researched |
Factual |
18 Feb 2015 |
The architecture of convolutional nets is inspired by basic knowledge of the visual cortex.
Factual · 18 Feb 2015 |
AI capabilities |
| Not researched |
Factual |
18 Feb 2015 |
Unsupervised techniques can be beaten by collecting more data and using supervised learning.
Factual · 18 Feb 2015 |
AI capabilities |
| Not researched |
Factual |
15 May 2014 |
SVMs do not naturally emerge from VC theory; they are simple enough for VC theory to give specific results about them.
Factual · 15 May 2014 |
Science and research methods |