ClaimsYann LeCun

“Very often, the inference algorithm can only give us an approximate answer, or is not guaranteed to give us the global minimum of the energy.”

Yann LeCun et al., tutorial authors 2006 A Tutorial on Energy-Based Learning (yann.lecun.com) archived copy

Factual AI capabilities

Inference algorithms in energy-based learning often give only approximate answers or do not guarantee finding the global energy minimum.

Resolved true The record shows it happened or was so.

Factual: Statements about the present or the past, checked against the record: did they hold up?

The second judge would not resolve this claim; the verdict shown is the first judge's.

What would count as true or false

Judge as of 2006 against the energy-based learning and optimisation literature of that time. True: standard sources (textbooks, surveys, the energy-based learning tutorial literature) show that in many practical energy-based models the inference step (minimising energy over outputs or latent variables) is done with approximate methods or with methods that can get stuck in local minima, because exact global minimisation is intractable or non-convex in general. Mostly true: this holds in substance, but 'very often' overstates it slightly, for example if exact inference was in fact typical for the model classes in common use. Mixed: approximate or non-global inference was common for some major model classes but rare for others, so 'very often' holds for only about half of them. Mostly false: approximate or non-global inference happened only in a narrow set of cases and exact global inference was the norm. False: the evidence shows that inference in energy-based models of that period was generally exact and guaranteed to find the global minimum.

Set before the research began.

Evidence (4)

  1. E1 Markov Random Fields University of Toronto, CSC 2503 course handout (D.J. Fleet & A.D. Jepson) · 2011

    “Inference with MRFs is challenging as useful factorizations of the joint distribution, like those for chains and trees are not available. For all but a few special cases, MAP estimation is NP-hard.”
  2. E2 Markov Random Field Modeling, Inference & Learning in Computer Vision & Image Understanding: A Survey Computer Vision and Image Understanding (Wang, Komodakis, Paragios), archived on HAL · 2013

    “The minimization of such non-submodular energy functions is NP-hard in general [23, 25] and an approximation algorithm would be required to approach the global optimum.”
  3. E3 Energy-Based Models cs.nyu.edu/~yann (LeCun's NYU research page) · 19 July 2005 Own record

    “Again, no collapse occurs, but the computation is considerably more expensive than in the previous example, because we need to compute the integral over all values of Y”
  4. E4 Energy-Based Models and Loss Functions, Linear Machines Yann LeCun, NYU Courant Institute (course lecture, Spring 2004) · 2004 Own record

    “Best output: (cid:20)Y = minY E(W; Y; X) = W 0X.”

Reasoning

E1 says that MAP estimation in Markov random fields is NP-hard "for all but a few special cases", and E2 says that, in the same MRF setting, minimising non-submodular energy functions is NP-hard in general. E2 adds that an approximation algorithm would be required to approach the global optimum. Both sources date from after 2006 (2011 and 2013), but they state general complexity results that do not depend on the date, so they support the claim as of 2006. E4 shows a linear model where exact minimisation is trivial, but that is the kind of special case E1 sets aside, and it does not undercut "very often".

Cite this claim

Yann LeCun, “Inference algorithms in energy-based learning often give only approximate answers or do not guarantee finding the global energy minimum.” (said 2006). Verdict: resolved true. Reliability API, claim c_8c8c26210cba, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_8c8c26210cba

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Yann LeCun, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More factual claims by Yann LeCun

  1. Mostly true Factual 18 March 2024
    “For starters, transformers are equivariant to permutations.”

    Transformers are equivariant to permutations, as ConvNets are equivariant to translations.

  2. Mostly true Factual 14 March 2024
    “If it were the case, we would have AI systems that could teach themselves to drive a car in 20 hours of practice, like any 17 year-old.”

    There are currently no AI systems that can teach themselves to drive a car in about 20 hours of practice, as a 17-year-old can.

  3. Mostly true Factual 8 March 2024
    “The architecture is good, the architecture of the encoder is good, but the fact that you train the system to reconstruct images does not lead it to produce to long, good generic features of images.”

    Training an encoder to reconstruct images does not lead it to produce good generic image features, though the encoder architecture is good.

  4. Mostly true Factual 8 March 2024
    “LLMs can do none of those or they can only do them in a very primitive way and they don’t really understand the physical world.”

    Current LLMs can do none of understanding the physical world, persistent memory, reasoning and planning, or only in a very primitive way.

  5. Mostly true Factual 8 March 2024
    “No, there’s one thing that autoregressive LLMs or that LLMs in general, not just the autoregressive one, but including the bird style bidirectional ones, are exploiting and its self supervised…”

    LLMs, including BERT-style bidirectional ones, exploit self-supervised learning, and LeCun has been a very strong advocate of self-supervised learning for many years.

All of them

Verdicts are made automatically; how verdicts are made.