Inference algorithms in energy-based learning often give only approximate answers or do not guarantee finding the global energy minimum.
Resolved true The record shows it happened or was so.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judge as of 2006 against the energy-based learning and optimisation literature of that time. True: standard sources (textbooks, surveys, the energy-based learning tutorial literature) show that in many practical energy-based models the inference step (minimising energy over outputs or latent variables) is done with approximate methods or with methods that can get stuck in local minima, because exact global minimisation is intractable or non-convex in general. Mostly true: this holds in substance, but 'very often' overstates it slightly, for example if exact inference was in fact typical for the model classes in common use. Mixed: approximate or non-global inference was common for some major model classes but rare for others, so 'very often' holds for only about half of them. Mostly false: approximate or non-global inference happened only in a narrow set of cases and exact global inference was the norm. False: the evidence shows that inference in energy-based models of that period was generally exact and guaranteed to find the global minimum.
Set before the research began.
Evidence (4)
-
E1 Markov Random Fields University of Toronto, CSC 2503 course handout (D.J. Fleet & A.D. Jepson) · 2011
“Inference with MRFs is challenging as useful factorizations of the joint distribution, like those for chains and trees are not available. For all but a few special cases, MAP estimation is NP-hard.”
-
E2 Markov Random Field Modeling, Inference & Learning in Computer Vision & Image Understanding: A Survey Computer Vision and Image Understanding (Wang, Komodakis, Paragios), archived on HAL · 2013
“The minimization of such non-submodular energy functions is NP-hard in general [23, 25] and an approximation algorithm would be required to approach the global optimum.”
-
E3 Energy-Based Models cs.nyu.edu/~yann (LeCun's NYU research page) · 19 July 2005 Own record
“Again, no collapse occurs, but the computation is considerably more expensive than in the previous example, because we need to compute the integral over all values of Y”
-
E4 Energy-Based Models and Loss Functions, Linear Machines Yann LeCun, NYU Courant Institute (course lecture, Spring 2004) · 2004 Own record
“Best output: (cid:20)Y = minY E(W; Y; X) = W 0X.”
Reasoning
E1 says that MAP estimation in Markov random fields is NP-hard "for all but a few special cases", and E2 says that, in the same MRF setting, minimising non-submodular energy functions is NP-hard in general. E2 adds that an approximation algorithm would be required to approach the global optimum. Both sources date from after 2006 (2011 and 2013), but they state general complexity results that do not depend on the date, so they support the claim as of 2006. E4 shows a linear model where exact minimisation is trivial, but that is the kind of special case E1 sets aside, and it does not undercut "very often".
Cite this claim
Yann LeCun, “Inference algorithms in energy-based learning often give only approximate answers or do not guarantee finding the global energy minimum.” (said 2006). Verdict: resolved true. Reliability API, claim c_8c8c26210cba, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_8c8c26210cba
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Yann LeCun
-
Mostly true
“For starters, transformers are equivariant to permutations.”
Transformers are equivariant to permutations, as ConvNets are equivariant to translations.
-
Mostly true
“If it were the case, we would have AI systems that could teach themselves to drive a car in 20 hours of practice, like any 17 year-old.”
-
Mostly true
“The architecture is good, the architecture of the encoder is good, but the fact that you train the system to reconstruct images does not lead it to produce to long, good generic features of images.”
-
Mostly true
“LLMs can do none of those or they can only do them in a very primitive way and they don’t really understand the physical world.”
-
Mostly true
“No, there’s one thing that autoregressive LLMs or that LLMs in general, not just the autoregressive one, but including the bird style bidirectional ones, are exploiting and its self supervised…”
Verdicts are made automatically; how verdicts are made.