| Not checkable |
Not checkable |
8 Mar 2024 |
The idea somehow that we can’t get it slightly wrong because if we get it slightly wrong, we’ll die is ridiculous.
Not checkable · 8 Mar 2024 |
|
| Not checkable |
Not checkable |
8 Mar 2024 |
So there’s going to be various versions of LLaMA that are improvements of previous LLaMAs, bigger, better, multimodal, things like that.
Not checkable · 8 Mar 2024 |
|
| Not researched |
Factual |
8 Mar 2024 |
Everything humans learn in the first few years of life, and everything animals learn, has nothing to do with language.
Factual · 8 Mar 2024 |
AI capabilities |
| Unresolvable |
Prediction |
8 Mar 2024 |
Progress toward human-level AI will be gradual.
Prediction · 8 Mar 2024 |
AI timelines and forecasting |
| Unresolvable |
Factual |
28 Jun 2023 |
MIT Technology Review published a piece by Melissa Heikkilä about Meta's stance on AI existential risks, with quotes from Joelle Pineau and Yann LeCun.
Factual · 28 Jun 2023 |
AI safety and alignment |
| Not researched |
Factual |
28 Jun 2023 |
Data center power consumption barely changes because it is largely determined by economics: Google and Meta cannot spend more than a few dollars per user per year on power.
Factual · 28 Jun 2023 |
Economics and markets |
| Mostly true |
Factual |
28 Jun 2023 |
The US is a clear outlier among developed countries in healthcare cost and outcomes.
Factual · 28 Jun 2023 |
Other |
| Mostly true |
Factual |
28 Jun 2023 |
The audience YES/NO split at the Munk Debate was 67/33 before the debate and 63/37 after.
Factual · 28 Jun 2023 |
AI safety and alignment |
| Mostly true |
Factual |
28 Jun 2023 |
In the official result of the Munk Debate on AI existential threat, the NO side (Mitchell and LeCun) won by 4% over the YES side (Tegmark and Bengio).
Factual · 28 Jun 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
28 Jun 2023 |
Also, my ears are not pointy, and my archer skills are feeble.
Not checkable · 28 Jun 2023 |
|
| Not checkable |
Not checkable |
27 Jun 2023 |
Nice piece. But it should have talked about @NablaTech one of the first players in that space (which the NYT wrote about a while back).
Not checkable · 27 Jun 2023 |
|
| Resolved true |
Factual |
27 Jun 2023 |
Clément Delangue defended AI open science and open source in his congressional testimony.
Factual · 27 Jun 2023 |
AI policy and governance |
| Not checkable |
Not checkable |
26 Jun 2023 |
We are missing a number of basic concepts to get human-level AI.
Not checkable · 26 Jun 2023 |
|
| Not checkable |
Not checkable |
26 Jun 2023 |
I also agree that many of the AI safety policies that have been floating around are naïve.
Not checkable · 26 Jun 2023 |
|
| Not checkable |
Not checkable |
26 Jun 2023 |
Good thread on AI policy from @jackclarkSF
Not checkable · 26 Jun 2023 |
|
| Not checkable |
Not checkable |
26 Jun 2023 |
I agree that limiting compute or requiring a license above a certain threshold of compute power would be bad policy.
Not checkable · 26 Jun 2023 |
|
| Not checkable |
Not checkable |
25 Jun 2023 |
Thanks for clarifying.
Not checkable · 25 Jun 2023 |
|
| Not checkable |
Not checkable |
25 Jun 2023 |
On le construira pour être bénéfique.
Not checkable · 25 Jun 2023 |
|
| Not checkable |
Not checkable |
23 Jun 2023 |
He doesn't seem worried about the probability that he will produce intelligent evil psychopaths.
Not checkable · 23 Jun 2023 |
|
| Resolved true |
Factual |
23 Jun 2023 |
Max Tegmark is a professor at MIT.
Factual · 23 Jun 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
23 Jun 2023 |
We can do the same with AI: we can thoroughly test their benevolence before we give them any knowledge and agency.
Not checkable · 23 Jun 2023 |
|
| Not researched |
Factual |
22 Jun 2023 |
A large number of Americans believe in conspiracy theories, though the proportion has not increased over time.
Factual · 22 Jun 2023 |
Philosophy, ethics and society |
| Not researched |
Factual |
22 Jun 2023 |
Yann LeCun gave an opening statement at the Munk Debate on the existential risk of AI on 2023-06-22.
Factual · 22 Jun 2023 |
AI safety and alignment |
| Not checkable |
Not checkable |
22 Jun 2023 |
Automated manufacturing makes handcrafted goods more valuable.
Not checkable · 22 Jun 2023 |
|
| Not checkable |
Not checkable |
21 Jun 2023 |
Fun CIFAR workshop, as usual.
Not checkable · 21 Jun 2023 |
|
| Unresolvable |
Prediction |
20 Jun 2023 |
Objective-Driven AI architectures are the future of AI.
Prediction · 20 Jun 2023 |
AI capabilities |
| Unresolvable |
Factual |
19 Jun 2023 |
A joint interview of Yoshua Bengio and Yann LeCun was published in Les Échos Weekend a few days before 2023-06-19.
Factual · 19 Jun 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
19 Jun 2023 |
I can't disagree!
Not checkable · 19 Jun 2023 |
|
| Not researched |
Factual |
18 Jun 2023 |
Meta AI / FAIR has a large presence at CVPR 2023.
Factual · 18 Jun 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
18 Jun 2023 |
How to shoot yourself in the foot with a bazooka in the form of insane immigration laws.
Not checkable · 18 Jun 2023 |
|
| Not researched |
Factual |
18 Jun 2023 |
Yann LeCun is an advisor to Enhance Lab.
Factual · 18 Jun 2023 |
Their own work or organisation |
| Not researched |
Factual |
15 Jun 2023 |
Yann LeCun expressed opinions on the future of AI at the Innovation Press Day held at Meta-Paris on Tuesday 2023-06-13.
Factual · 15 Jun 2023 |
Their own work or organisation |
| Not checkable |
Not checkable |
15 Jun 2023 |
A good article by the BBC about my views on the future of AI.
Not checkable · 15 Jun 2023 |
|
| Not researched |
Factual |
14 Jun 2023 |
Ania Dawid transcribed Yann LeCun's lectures on Energy-Based Models, which he gave at the Les Houches summer school in 2022.
Factual · 14 Jun 2023 |
Science and research methods |
| Not researched |
Factual |
Jun 2023 |
The denoising autoencoder (ref. 51) is a type of contrastive energy-based model.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
Supervised and reinforcement learning require an enormous number of labeled samples or trials.
Factual · Jun 2023 |
AI capabilities |
| Not checkable |
Not checkable |
Jun 2023 |
Human learning, in contrast, is often highly efficient.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
However, which information is important may be scale-dependent.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
The proposed solution is, therefore, to replace probabilistic models with energy-based models (EBMs) and combine them with latent variables…
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
In the case of the latent variable EBMs (Fig. 4(b)), such
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
Contrastive methods aim to push down the energy of training samples and to pull up the energy of suitably-generated contrastive samples…
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
Current automated systems have crucial limitations that need to be addressed before artificial intelligence can reach human-like levels and…
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
An important feature of latent variable models is that while they are composed of deterministic functions, they are non-deterministic as…
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
Therefore, the energy function is not the objective function to minimize within the learning!
Not checkable · Jun 2023 |
|
| Unresolvable |
Factual |
Jun 2023 |
Contrastive methods for training energy-based models work for any architecture but scale badly with data dimension.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
Energy-based models can represent multimodal dependencies in which multiple values of y are compatible with a given x.
Factual · Jun 2023 |
AI capabilities |
| Mostly true |
Factual |
Jun 2023 |
Masked autoencoders are less successful with images and other continuous domains than with discrete domains such as language.
Factual · Jun 2023 |
AI capabilities |
| Unresolvable |
Prediction |
Jun 2023 |
H-JEPA trained with regularized methods may be a starting point for predictive world models capable of hierarchical planning under uncertainty, a breakthrough for autonomous AI.
Prediction · Jun 2023 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
Jun 2023 |
This proposal is important to the whole ML community
Not checkable · Jun 2023 |
|
| Not researched |
Factual |
Jun 2023 |
Contrastive methods for training energy-based models tend to become very inefficient in high dimensions.
Factual · Jun 2023 |
AI capabilities |
| Not checkable |
Not checkable |
Jun 2023 |
SSL is the cake (provides millions of information bits per sample)
Not checkable · Jun 2023 |
|
| Not researched |
Citation |
Jun 2023 |
Convolutional sparse autoencoders learn features corresponding to known image filters used for feature extraction and compressed representation (ref. 47).
Citation · Jun 2023 |
AI capabilities |
| Not checkable |
Not checkable |
Jun 2023 |
Firstly, it is challenging to understand uncertainty in an energy-based setting.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
To prevent the energy collapse, i.e., flat energy landscape, we need to constrain our energy surface so that data points have low energies…
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
A joint embedding predictive architecture (JEPA), presented in Fig. 9, is an EBM that combines
Not checkable · Jun 2023 |
|
| Unresolvable |
Prediction |
Jun 2023 |
Self-supervised learning is the most likely approach to learning world representations, since supervised and reinforcement learning require too many samples or trials.
Prediction · Jun 2023 |
AI capabilities |
| Not checkable |
Not checkable |
Jun 2023 |
Energy-based models (EBMs) capture dependencies between variables x and y by associating a scalar energy F (x, y) to each configuration of…
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
This pitfall is due to the folded structure of the data, which, however, rarely
Not checkable · Jun 2023 |
|
| Not researched |
Factual |
Jun 2023 |
Babies learn almost entirely by observation.
Factual · Jun 2023 |
Science and research methods |
| Not checkable |
Not checkable |
Jun 2023 |
it can collapse because it can learn the identity function and have zero energy for all the inputs.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
Such multi-level predictions may be achieved with a hierarchical joint embedding predictive architecture (H-JEPA).
Not checkable · Jun 2023 |
|
| Not researched |
Factual |
Jun 2023 |
Hopfield networks, fully-connected recurrent networks, were popularized by John Hopfield in 1982.
Factual · Jun 2023 |
AI capabilities |
| Not checkable |
Not checkable |
Jun 2023 |
The tricky part is that the information capacity of the latent variable must be minimized.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
Moreover, JEPAs naturally handle
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
The main disadvantage of maximum likelihood as a contrastive EBM is that it wants to make the difference between the energy on the data…
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
To avoid this drawback, we need to design an architecture that does not require transferring outputs between its components.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
While it may be surprising that we give up on a probabilistic setting, note that making decisions can be viewed as choosing the option with…
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
In particular, energies are uncalibrated (i.e., measured in arbitrary units), so combining two separately trained EBMs is not…
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
Finally, the maximum likelihood can be interpreted as a special case of a contrastive method.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
It means that the minimization of LNLL consists in minimizing the energy of training samples while pulling up the energy of Monte…
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
Moreover, this representation should be task-agnostic.
Not checkable · Jun 2023 |
|
| Unresolvable |
Prediction |
Jun 2023 |
Regularized methods seem more promising than contrastive methods for training future energy-based models, because they are less likely to fall victim to the curse of dimensionality.
Prediction · Jun 2023 |
AI timelines and forecasting |
| Not checkable |
Not checkable |
Jun 2023 |
However, in decision-making tasks such as driving a car, it is merely necessary that the system chooses the correct answer.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
Ideally, the latent variable represents independent explanatory factors of prediction variation.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
Therefore, the JEPA can be trained with a loss function that apart from the
Not checkable · Jun 2023 |
|
| Unresolvable |
Factual |
Jun 2023 |
Ideas related to VICReg-style regularization were explored already in 1992, per reference [62].
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
Joint embedding predictive architectures (JEPAs) are a new architecture proposed by LeCun in Ref. [6].
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Citation |
Jun 2023 |
Normalizing scores in energy-based models runs into the label bias problem (ref. 39), where transition probabilities out of a state are normalized per state rather than globally.
Citation · Jun 2023 |
AI capabilities |
| Not checkable |
Not checkable |
Jun 2023 |
Moreover, the modern ML systems are very specialized (trained for one task) and brittle (make stupid mistakes).
Not checkable · Jun 2023 |
|
| Not researched |
Factual |
Jun 2023 |
Supervised and reinforcement learning dominate modern real-world applications and are highly inefficient compared to human learning.
Factual · Jun 2023 |
AI capabilities |
| Mostly true |
Factual |
Jun 2023 |
Probabilistic models require normalization and may become intractable for high-dimensional data.
Factual · Jun 2023 |
AI capabilities |
| Not researched |
Factual |
Jun 2023 |
SSL for images produces blurry predictions.
Factual · Jun 2023 |
AI capabilities |
| Not checkable |
Not checkable |
Jun 2023 |
The key difference is that the world model predicting the future here is learned.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
While the modern large language models sometimes provide an illusion of reasoning, they have no understanding of reality.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
RL works great with games because they can be played over and over in parallel.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
Another approach is to add a regularization term that minimizes a certain measure
Not checkable · Jun 2023 |
|
| Not researched |
Citation |
Jun 2023 |
Score matching (ref. 46) is a regularization technique that minimizes the gradient and maximizes the curvature of the energy landscape around data points.
Citation · Jun 2023 |
AI capabilities |
| Not checkable |
Not checkable |
Jun 2023 |
Eventually, we hope to achieve a fully autonomous AI, which can perform well on generic tasks by transferring its knowledge and…
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
The problem of such
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
The reason for the limited success of masked AEs in continuous domains like images or videos
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
regularized methods seem more promising for training EBMs of the future.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
All in all, every model that can be multimodal, i.e., have multiple predictions for a single input, is susceptible to collapse.
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
Negative loglikelihood is then a contrastive method!
Not checkable · Jun 2023 |
|
| Not researched |
Factual |
Jun 2023 |
Most large-scale machine learning applications today rely on supervised learning or deep reinforcement learning, which require extreme computational expense.
Factual · Jun 2023 |
AI compute, hardware and data |
| Not researched |
Factual |
Jun 2023 |
Boltzmann machines were fashionable for some time, but the large cost of sampling makes them less useful for real-world applications.
Factual · Jun 2023 |
AI capabilities |
| Not checkable |
Not checkable |
Jun 2023 |
Perhaps surprisingly, this representation encompasses many constrained fitting
Not checkable · Jun 2023 |
|
| Not researched |
Factual |
Jun 2023 |
As of mid-2023, there are no Level 5 self-driving cars, domestic robots, or virtual assistants that learn reliable world models, reason and plan complex action sequences.
Factual · Jun 2023 |
AI capabilities |
| Not checkable |
Not checkable |
Jun 2023 |
This limitation renders the Hopfield networks less
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
In particular, to prevent the informational energy collapse, we need to make sure that sx and s y carry
Not checkable · Jun 2023 |
|
| Not checkable |
Not checkable |
Jun 2023 |
Discrete and continuous Gibbs distribution (or softmax, should be called softargmax):
Not checkable · Jun 2023 |
|