| Not checkable |
Not checkable |
2006 |
Non-linear factors allow the modeling of highly complex dependencies between inputs and labels
Not checkable · 2006 |
|
| Not researched |
Factual |
2006 |
The dependency between X and Y sometimes cannot be expressed as a function mapping X to Y, e.g. the constraint X^2+Y^2=1.
Factual · 2006 |
AI capabilities |
| Not checkable |
Not checkable |
2006 |
Probabilistic models have two major disadvantages.
Not checkable · 2006 |
|
| Not checkable |
Not checkable |
2006 |
In the energybased framework, there is no need to make excuses for breaking the rules.
Not checkable · 2006 |
|
| Not checkable |
Not checkable |
2006 |
This loss function can be optimized with a variety of techniques.
Not checkable · 2006 |
|
| Not researched |
Factual |
2006 |
With the NLL loss the minimum is deeper than with the square-square loss because incorrect answers' energies are pushed up to infinity, with decreasing force.
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
Inference by marginalization over latent variables reduces to inference by minimization when β→∞ (zero temperature).
Factual · 2006 |
AI capabilities |
| Resolved true |
Factual |
2006 |
Each NLL iteration involves considerably more work because pushing up every incorrect answer is expensive when the second term's derivative has no analytical form.
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
In 2006, the authors said that, to their knowledge, no systematic experimental study had been published on whether second-order methods bring a significant speed improvement over well-tuned stochastic gradient methods for training these models.
Factual · 2006 |
AI capabilities |
| Not checkable |
Not checkable |
2006 |
Energy-based learning can be seen as an alternative to probabilistic estimation for prediction, classification, or decision-making tasks.
Not checkable · 2006 |
|
| Not researched |
Factual |
2006 |
Because energies are uncalibrated, there is no a priori guarantee that separately trained energy-based models have commensurate energy scales.
Factual · 2006 |
Science and research methods |
| Not checkable |
Not checkable |
2006 |
In this way, the regression on X1 and X2 is implicitly learned through the constraint Y rather than explicitly learned through supervision.
Not checkable · 2006 |
|
| Not checkable |
Not checkable |
2006 |
Many existing learning models can be be expressed simply in the framework of energy-based learning.
Not checkable · 2006 |
|
| Not checkable |
Not checkable |
2006 |
What we propose here is to view the probabilistic approach as a special case of a much larger family of energy-based methods.
Not checkable · 2006 |
|
| Not researched |
Factual |
2006 |
One of the alleged advantages of CRFs is that their loss function is convex with respect to the parameters W.
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
The main problem with the perceptron loss is the absence of a margin.
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
Because of its zero margin, the generalized perceptron loss could lead to a collapsed energy surface.
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
The log loss can be seen as a soft version of the hinge loss with an infinite margin.
Factual · 2006 |
Science and research methods |
| Not checkable |
Not checkable |
2006 |
Properly designing the architecture and the loss function is critical.
Not checkable · 2006 |
|
| Not checkable |
Not checkable |
2006 |
Such hidden variables are called latent variables, by analogy with a similar concept in probabilistic modeling.
Not checkable · 2006 |
|
| Not researched |
Factual |
2006 |
Learning and inference with EBMs involve minimizing the energy over the set of answers Y and latent variables Z.
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
When the energy is linear in W, the hinge loss with L2 regularizer takes the simple form of Eq. 81.
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
The energy and loss functions must satisfy the stated sufficient conditions to be guaranteed to work in an energy-based setting.
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
The NLL loss reduces to the generalized perceptron loss as β→∞, and to the log loss when Y has two elements.
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
The square-exponential loss has an infinite margin and pushes incorrect answers' energies to infinity with exponentially decreasing force.
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
Siamese architectures are used to learn similarity metrics with labeled examples.
Factual · 2006 |
AI capabilities |
| Not checkable |
Not checkable |
2006 |
Graph transformer networks are hierarchical sequence modeling systems in which the objects that are manipulated are trellises containing…
Not checkable · 2006 |
|
| Not checkable |
Not checkable |
2006 |
For other models, one cannot guarantee anything.
Not checkable · 2006 |
|
| Not researched |
Factual |
2006 |
Maximizing the conditional likelihood of training samples is equivalent to minimizing the negative log-likelihood loss.
Factual · 2006 |
AI capabilities |
| Unresolvable |
Factual |
2006 |
There exists at least one point in HP1 at which the loss is less than at all points in HP2.
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
With a linear parameterization, training with the energy loss reduces to a convex least-squares problem.
Factual · 2006 |
AI capabilities |
| Not checkable |
Not checkable |
2006 |
Energy-based methods are equally well justified as probabilistic methods.
Not checkable · 2006 |
|
| Not researched |
Factual |
2006 |
The energy of the most offending incorrect answer is bounded above by the energy of the next most offending incorrect answer.
Factual · 2006 |
AI capabilities |
| Not checkable |
Not checkable |
2006 |
The architectures described in the previous section are simple functions of Y with a single minimum within the set Y.
Not checkable · 2006 |
|
| Not checkable |
Not checkable |
2006 |
This could be construed as a thinly disguised way of removing the normalization constraint.
Not checkable · 2006 |
|
| Not researched |
Factual |
2006 |
Normalization is only necessary for generative models.
Factual · 2006 |
AI capabilities |
| Not checkable |
Not checkable |
2006 |
These models can be easily described in an EBM setting.
Not checkable · 2006 |
|
| Not researched |
Factual |
2006 |
When the RBF centers are learned rather than fixed, there is no guarantee that d1 + d2 ≥ d.
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
Section 5 of this tutorial concluded that the generalized perceptron loss is not a good loss function.
Factual · 2006 |
AI capabilities |
| Not checkable |
Not checkable |
2006 |
The EBM approach provides a common theoretical framework for many learning models, including traditional discriminative and generative…
Not checkable · 2006 |
|
| Not researched |
Factual |
2006 |
Energy-based models have no requirement for proper normalization, so they avoid the problem of evaluating intractable normalizing integrals.
Factual · 2006 |
Science and research methods |
| Not researched |
Factual |
2006 |
The cost of shortest-path inference is proportional to the number of edges, which is exponentially smaller than the number of paths in general.
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
Any probability distribution over Y can be approximated arbitrarily closely by a Gibbs distribution of the form e^(-βE)/Σe^(-βE).
Factual · 2006 |
AI capabilities |
| Not researched |
Factual |
2006 |
The EBM approach eliminates the problem of having to perform a normalization step after each gradient parameter update (citing LeCun et al., 1998a). 1998a.
Factual · 2006 |
Their own work or organisation |
| Not checkable |
Not checkable |
2005 |
Face recognition: the gender of the person, the orientation of the face.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
MAP Inference: Given an input X, find the value of Y that minimizes the energy:
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Variables that would make the task easier if they were known:
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Do not answer a more complex question than necessary
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Minimize the Regularization term: Make W as short as possible.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
This is equivalent to keeping ||W|| constant, while maximizing m.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Minimize the hinge loss: make the energy of all the “good” answers smaller that the energy of any “bad” answer by at least m (the margin).
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Probabilistic approaches compute the distribution P(Y|X), and choose the Y that maximizes it.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Computing a distribution of steering angles is of little use.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
W is the parameter to be learned.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
For each question, a different learning strategy is required
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Pick an optimization method.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Linear Model + Margin Loss + Regularization = SVM
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
E(W,Y,X): is a scalar energy function (a.k.a. Contrast function) that measures the “compatibility” between Y and X.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Object recognition: the pose parameters of the object (location, orientation, scale), the lighting conditions.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Handwriting Recognition: the segmentation of the line into characters.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Design an architecture: a particular form for E(W,Y,X).
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Probabilistic Inference: simply marginalize over Z.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
A factor graph is a general way to represent a graphical model.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
For decision making, we need no normalization.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Sometimes, we do not need probabilities.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
The internal structure of the box is called the architecture of the model.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
A model measures the goodness of a combination of observed variables (X) and variables to be predicted (Y).
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
The questions are in increasing order of complexity.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
We can put anything we want in the box.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Architecture + Inference Algo + Loss Function = Model
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
In general, we will search for the value of the latent variable that allows us to get an answer (Y) of smallest energy.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
The machine should be designed and trained to answer the simplest question possible.
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Z: is the latent variable (never observed).
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
Pick a loss function: in such a way that minimizing it with respect to W over a training set will make the inference algorithm find the…
Not checkable · 2005 |
|
| Not checkable |
Not checkable |
2005 |
When the robot faces an obstacle, it MUST turn left of right.
Not checkable · 2005 |
|