| Not researched |
Michael Burry | Factual |
2006 |
In the example subprime pool, senior tranches A-1A and A-1B make up 79% of the pool and have credit support of 21% of the pool.
Michael Burry · Factual · 2006 |
Economics and markets |
| Not researched |
Yann LeCun | Factual |
2006 |
EBMs manipulate energies, so no normalization is necessary.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Because the MEE loss saturates the contribution of any single error, it is more robust to label noise and outliers but is non-convex.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The lack of a margin in the perceptron loss, which may theoretically cause stability problems, was overlooked in LeCun et al. 1998a.
Yann LeCun · Factual · 2006 |
Their own work or organisation |
| Not researched |
Yann LeCun | Factual |
2006 |
The quality of an answer in an energy-based model cannot be measured by its energy alone without considering the energies of all other answers.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
For E(Y,X)=Y·Gface(X)+(1−Y)T, the minimizing Y is 1 if Gface(X)<T and 0 otherwise.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Inference in the described factor graph can be reduced to finding the shortest path in the graph.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The generalized perceptron loss has a margin of zero.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
In the NLL loss gradient, the contrastive term pulls up each answer's energy with a force proportional to that answer's likelihood under the model.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
A collapsed energy surface can be avoided by including a contrastive term in the loss function.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The perceptron loss and the LVQ2 loss satisfy condition 3 with a margin of zero.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
A meaningful, uncollapsed result with the perceptron loss is only guaranteed if the model cannot produce a flat energy surface.
Yann LeCun · Factual · 2006 |
Science and research methods |
| Not researched |
Yann LeCun | Factual |
2006 |
This loss function, though popular for regression and neural network training, cannot be used to train most architectures, because it pushes down on the energy of the desired answer without pulling up on any other energy.
Yann LeCun · Factual · 2006 |
Science and research methods |
| Not researched |
Yann LeCun | Factual |
2006 |
The min-sum procedure only applies to factor graphs that are bipartite trees with no loops.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The partition function over configurations of a loopy graphical model cannot be computed exactly, but loopy belief propagation or other variational methods may approximate it.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The negative log-likelihood loss for structured models requires summing over all label combinations, which number 2^l for a sequence of l binary labels.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The square-square loss is only suitable for energy functions bounded below by zero, because it pins the correct answer's energy at zero.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
In the tutorial's first experiment, the architecture's constraints automatically raise the energy of undesired answers while lowering that of desired answers.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
As long as d ≥ m, the loss function shown in Figure 16(b) satisfies condition 3.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The energy loss only works with architectures where pushing down the correct answer's energy automatically raises the energies of other answers.
Yann LeCun · Factual · 2006 |
Science and research methods |
| Not researched |
Michael Burry | Factual |
2006 |
Burry said that as 2005 came to a close, organizers of new mortgage pools included or extended use of extra protections to bolster credit support for the pools' tranches, and that this is why he found many more recent deals much less attractive from a short's perspective than mid-2005 deals.
Michael Burry · Factual · 2006 |
Economics and markets |
| Not researched |
Yann LeCun | Factual |
2006 |
The generalized margin loss functions discussed satisfy condition 3.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Energy-based loss functions that pull up the most offending incorrect answer pull up only a single energy per learning iteration.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Michael Burry | Factual |
2006 |
In the PPSI 2005-WLL1 deal, Argent Mortgage and Olympus Mortgage separately originated subprime mortgages and each sold them to Ameriquest Mortgage Company.
Michael Burry · Factual · 2006 |
Economics and markets |
| Not researched |
Yann LeCun | Factual |
2006 |
In the NLL loss, the contrastive term pulls up the energies of all candidate answers.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
In the two-class problem, E_C and E_I correspond directly to the energies of the two classes.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The negative log-likelihood loss can only be used if the exponential of the negative energy is integrable over Y.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The preceding discussion of EBM inference and learning did not require manipulating normalized probability distributions.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Citation |
2006 |
Huang and LeCun (2006) and Collobert et al. (2006) argued that convex loss functions do not guarantee good performance and that non-convex losses may be easier to optimize in practice.
Yann LeCun · Citation · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The hinge-type loss's second term prevents energy collapse by pushing up points whose energy threatens to go below that of the desired answer.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
For the negative log-likelihood loss, the gradient with respect to E_C is positive and with respect to E_I is negative for any energy values.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The probability of training hitting a collapsed solution with the perceptron loss is quite small.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Since the RBF centers U1 and U2 are fixed and distinct, there is a strictly positive lower bound on d1 + d2 for all G_W.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
For the squared-error regression energy, the minimum energy is always zero.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
With this simple architecture, the NLL or perceptron loss would give exactly the same result as the energy loss because their contrastive term is constant.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Local minima are rarely a problem in practice for gradient-based learning because of the high dimensionality of the space.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Stochastic gradient methods can be applied to any loss function, including nonconvex ones.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
In many practical situations computing the partition function is intractable or impossible, such as when Y has high cardinality or the integral has no analytical solution.
Yann LeCun · Factual · 2006 |
Science and research methods |
| Not researched |
Yann LeCun | Factual |
2006 |
Hinge, log, LVQ2, MCE, square-square and square-exponential losses can all be described as margin losses.
Yann LeCun · Factual · 2006 |
Science and research methods |
| Not researched |
Michael Burry | Factual |
2006 |
Home prices did not appreciate during the year before this 2006 letter.
Michael Burry · Factual · 2006 |
Economics and markets |
| Not researched |
Yann LeCun | Factual |
2006 |
With the energy loss, nothing prevents G1 and G2 from ignoring their inputs and producing identical outputs, causing a collapsed flat energy surface.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The set of collapsed solutions is a small piece of the parameter space.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
A carefully tuned stochastic gradient descent is often considerably faster in practice than sophisticated second-order methods because it exploits redundancy between samples.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Citation |
2006 |
Hadsell et al. (2006) used Siamese architectures with the square-square loss for unsupervised learning of manifolds.
Yann LeCun · Citation · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The loss function for the highly non-linear GTN-type parameterization is not convex in W.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
With some architectures, the energy loss can lead to a collapsed solution in which the energy is constant and equal to zero.
Yann LeCun · Factual · 2006 |
Science and research methods |
| Not researched |
Yann LeCun | Factual |
2006 |
Discriminative training often uses iterative gradient-based methods to optimize the loss.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
For many interesting models, computing the integral over Y in the NLL loss is intractable.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
In the third and fourth experiments, using loss functions with explicit contrastive terms, the machine performed as expected and did not collapse.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Unlike generalized margin losses, the LVQ2 loss is non-convex in the correct and most offending incorrect energies.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
As of 2006, GTNs had never been trained with a generalized margin loss, to the authors' knowledge.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Probabilistic learning is a special case of energy-based learning in which the loss function is the negative log-likelihood, also called the maximum mutual information criterion.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
For the squared-error regression energy, the Y minimizing E equals G_W(X).
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Excellent results were obtained for isolated word recognition with the LVQ2 loss despite its zero margin.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Michael Burry | Factual |
2006 |
Credit default swap protection contracts on these tranches are pay-as-you-go.
Michael Burry · Factual · 2006 |
Economics and markets |
| Not researched |
Yann LeCun | Factual |
2006 |
As of 2006, no theory exists for energy-based learning with approximate inference.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The NLL loss is invariant to global shifts of energy values and depends only on energy differences among the Ys for a given X.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The dependency between X and Y sometimes cannot be expressed as a function mapping X to Y, e.g. the constraint X^2+Y^2=1.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
With the NLL loss the minimum is deeper than with the square-square loss because incorrect answers' energies are pushed up to infinity, with decreasing force.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Inference by marginalization over latent variables reduces to inference by minimization when β→∞ (zero temperature).
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
In 2006, the authors said that, to their knowledge, no systematic experimental study had been published on whether second-order methods bring a significant speed improvement over well-tuned stochastic gradient methods for training these models.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Because energies are uncalibrated, there is no a priori guarantee that separately trained energy-based models have commensurate energy scales.
Yann LeCun · Factual · 2006 |
Science and research methods |
| Not researched |
Yann LeCun | Factual |
2006 |
One of the alleged advantages of CRFs is that their loss function is convex with respect to the parameters W.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The main problem with the perceptron loss is the absence of a margin.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Because of its zero margin, the generalized perceptron loss could lead to a collapsed energy surface.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The log loss can be seen as a soft version of the hinge loss with an infinite margin.
Yann LeCun · Factual · 2006 |
Science and research methods |
| Not researched |
Michael Burry | Factual |
2006 |
The secondary market for mortgages is vast and deep thanks to the invention of mortgage-backed securitizations in the 1970s.
Michael Burry · Factual · 2006 |
Economics and markets |
| Not researched |
Yann LeCun | Factual |
2006 |
Learning and inference with EBMs involve minimizing the energy over the set of answers Y and latent variables Z.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
When the energy is linear in W, the hinge loss with L2 regularizer takes the simple form of Eq. 81.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The energy and loss functions must satisfy the stated sufficient conditions to be guaranteed to work in an energy-based setting.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The NLL loss reduces to the generalized perceptron loss as β→∞, and to the log loss when Y has two elements.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The square-exponential loss has an infinite margin and pushes incorrect answers' energies to infinity with exponentially decreasing force.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Siamese architectures are used to learn similarity metrics with labeled examples.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Maximizing the conditional likelihood of training samples is equivalent to minimizing the negative log-likelihood loss.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
With a linear parameterization, training with the energy loss reduces to a convex least-squares problem.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The energy of the most offending incorrect answer is bounded above by the energy of the next most offending incorrect answer.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Normalization is only necessary for generative models.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
When the RBF centers are learned rather than fixed, there is no guarantee that d1 + d2 ≥ d.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Section 5 of this tutorial concluded that the generalized perceptron loss is not a good loss function.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Energy-based models have no requirement for proper normalization, so they avoid the problem of evaluating intractable normalizing integrals.
Yann LeCun · Factual · 2006 |
Science and research methods |
| Not researched |
Yann LeCun | Factual |
2006 |
The cost of shortest-path inference is proportional to the number of edges, which is exponentially smaller than the number of paths in general.
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
Any probability distribution over Y can be approximated arbitrarily closely by a Gibbs distribution of the form e^(-βE)/Σe^(-βE).
Yann LeCun · Factual · 2006 |
AI capabilities |
| Not researched |
Yann LeCun | Factual |
2006 |
The EBM approach eliminates the problem of having to perform a normalization step after each gradient parameter update (citing LeCun et al., 1998a). 1998a.
Yann LeCun · Factual · 2006 |
Their own work or organisation |
| Not researched |
Demis Hassabis | Factual |
2004 |
Hassabis is executive designer on Evil Genius, meeting regularly with Sandy Sammarco and producer Pete Gilbert to set direction.
Demis Hassabis · Factual · 2004 |
Other |
| Not researched |
Demis Hassabis | Factual |
2004 |
Sandy Sammarco is lead designer of Evil Genius and heads the design day to day.
Demis Hassabis · Factual · 2004 |
Other |
| Not researched |
Demis Hassabis | Factual |
2004 |
Elixir Studios consciously chose Evil Genius as the game after Republic because it contrasted with Republic's seriousness.
Demis Hassabis · Factual · 2004 |
Other |
| Not researched |
Michael Burry | Factual |
2004 |
Inflation-adjusted oil prices have declined for decades and are not even half what they were in the 1970s.
Michael Burry · Factual · 2004 |
Economics and markets |
| Not researched |
Demis Hassabis | Factual |
2004 |
X-Com, Dungeon Keeper and Theme Park all feature simulation, missions and base building.
Demis Hassabis · Factual · 2004 |
Other |
| Not researched |
Ray Kurzweil | Factual |
12 Dec 2002 |
Human civilization's biological brains currently perform about 10^26 calculations per second in total.
Ray Kurzweil · Factual · 12 Dec 2002 |
AI compute, hardware and data |
| Not researched |
Ray Kurzweil | Factual |
12 Dec 2002 |
Computing price-performance now doubles in less than one year, versus three years early in the 20th century and two years mid-century.
Ray Kurzweil · Factual · 12 Dec 2002 |
AI compute, hardware and data |
| Not researched |
Ray Kurzweil | Factual |
12 Dec 2002 |
One cannot predict the outcome of a cellular-automaton-type process unless one can simulate the process.
Ray Kurzweil · Factual · 12 Dec 2002 |
Science and research methods |
| Not researched |
Ray Kurzweil | Factual |
12 Dec 2002 |
Computational capacity is currently being multiplied by a factor of at least 10^3 every decade.
Ray Kurzweil · Factual · 12 Dec 2002 |
AI compute, hardware and data |
| Not researched |
Ray Kurzweil | Factual |
12 Dec 2002 |
Machines can share their knowledge, for example by loading the learned software pattern of one computer into another.
Ray Kurzweil · Factual · 12 Dec 2002 |
AI capabilities |
| Not researched |
Ray Kurzweil | Factual |
12 Dec 2002 |
The Cambrian explosion established animal body plans that became a mature technology, so body plans did not need to evolve further.
Ray Kurzweil · Factual · 12 Dec 2002 |
Science and research methods |
| Not researched |
Ray Kurzweil | Factual |
12 Dec 2002 |
The SETI assumption is that although another Earth-like intelligent civilization is very unlikely, there are billions of trillions of planets.
Ray Kurzweil · Factual · 12 Dec 2002 |
Science and research methods |
| Not researched |
Ray Kurzweil | Factual |
12 Dec 2002 |
Magnetic data storage has very regularly doubled every year.
Ray Kurzweil · Factual · 12 Dec 2002 |
AI compute, hardware and data |
| Not researched |
Ray Kurzweil | Factual |
12 Dec 2002 |
Evolutionary processes accelerate and the returns from them grow in power (the Law of Accelerating Returns).
Ray Kurzweil · Factual · 12 Dec 2002 |
Technology |
| Not researched |
Ray Kurzweil | Factual |
12 Dec 2002 |
With DNA, evolution gained an information-processing machine to record its experiments and conduct them in a more orderly way.
Ray Kurzweil · Factual · 12 Dec 2002 |
Science and research methods |
| Not researched |
Warren Buffett | Factual |
28 Feb 2002 |
Underwriting discipline is being restored at General Re and its Cologne Re subsidiary.
Warren Buffett · Factual · 28 Feb 2002 |
Their own work or organisation |
| Not researched |
Warren Buffett | Factual |
28 Feb 2002 |
GEICO got into huge trouble in the early 1970s because for several years it severely underreserved.
Warren Buffett · Factual · 28 Feb 2002 |
Their own work or organisation |