Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
73 67 79 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
Insufficient record: 0 predictions resolved, 5 needed for a score.
-
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 0 promises resolved, 5 needed for a score.
All types combined: 73 (likely 67 to 79, high confidence), from 94 claims: 94 factual claims. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Nassim Taleb's claims
76 claims
“While this knowledge is well known in mathematical finance it doesn’t seem to have been passed on to the decision-theory literature.”
Nassim Taleb, paper author 2019 On the Statistical Differences between Binary Forecasts and Real World Payoffs (arXiv)
The difference between binary and vanilla payoffs is well known in mathematical finance but does not seem to have been passed on to the decision-theory literature.
Unresolvable The public record could not settle it.
What would count as true or false
The claim is hedged ('doesn't seem'). It is judged by whether the distinction between binary and vanilla payoffs, a standard idea in mathematical finance, had been taken up in the decision-theory literature before 2019. True: mathematical finance treats the distinction as standard, and reviews or searches of the decision-theory literature (for example on forecasting, probability judgement and expected utility) find it essentially absent. Mostly true: the distinction appears only in isolated decision-theory papers and is not a recognised topic in that field. Mixed: the distinction is discussed in a modest but real body of decision-theory work. Mostly false: decision theory addresses the distinction regularly under other names (for example the difference between probability and expected payoff), so the claim of non-transmission is overstated. False: decision-theory literature before 2019 treats the distinction explicitly and as standard, or mathematical finance does not treat it as well known.
Evidence
- FORECASTING AND DECISION THEORY - Mark Machina Handbook of Economic Forecasting, Vol. 1 (Elsevier), chapter by C.W.J. Granger and M.J. Machina · 2006
“Although many of the above results are well known in the Bayesian decision theory literature, they were less known in the forecasting area, where the use of the whole distribution rather than just the mean, and an economic cost function linked with a decision maker, were not usually emphasized.”
- [PDF] Exotic Derivatives - COPYRIGHTED MATERIAL Wiley (book excerpt, Advanced Equity Derivatives, Chapter 1)
“Strictly speaking, an exotic derivative is any derivative that is not a plain vanilla call or put.”
- On the Difference between Binary Prediction and True Exposure With Implications For Forecasting Tournaments and Decision Making Research Working paper by Nassim N. Taleb and Philip E. Tetlock (SSRN 2284964) · 2013 Own record
“Yet much of the economics and decision making literature confuses the two.”
- Forecasting and deciding binary outcomes under asymmetric information lenkiefer.com (blog by economist Len Kiefer) · 5 March 2018
“Where things get interesting is that we might suppose that the decider and the forecaster have different payoff function.”
- (none) - excerpt of Advanced Equity Derivatives, Chapter 1 'Exotic Derivatives' Wiley
“A European digital or binary option pays off $1 if the underlying asset price is above the strike K at maturity T, and 0 otherwise:”
- Chapter 2: Forecasting and Decision Theory Elsevier (Handbook of Economic Forecasting, Vol. 1) · 2006
“Although many of the above results are well known in the Bayesian decision theory literature, they were less known in the forecasting area, where the use of the whole distribution rather than just the mean, and an economic cost function linked with a decision maker, were not usually emphasized.”
- Chapter 2: Forecasting and Decision Theory Elsevier (Handbook of Economic Forecasting, Vol. 1) · 2006
“The case of hypothesis testing will correspond to the forecasting of binary events as illustrated in the above table”
- On the Difference between Binary Prediction and True Exposure With Implications For Forecasting Tournaments and Decision Making Research Not shown on page (working paper; SSRN abstract 2284964) · 2013 Own record
“Yet much of the economics and decision making literature confuses the two.”
- [PDF] A Course in Derivative Securities - ReadingSample Springer Finance (Kerry Back, A Course in Derivative Securities, 1st ed. 2005; reading sample via beck-shop.de) · 2005
“A digital (or “binary”) option pays a fixed amount in a certain event and zero otherwise.”
- [PDF] A Course in Derivative Securities - ReadingSample Springer Finance (Kerry Back, A Course in Derivative Securities, 1st ed. 2005; reading sample via beck-shop.de) · 2005
“As in Sect. 1.5, we will approach the pricing of call and put options by first considering their basic building blocks: digitals and share digitals.”
- The Binarized Scoring Rule of Belief Elicitation University of Toronto economics seminar file (Tanjim Hossain and Ryo Okui working paper) · October 2010
“Our method modifies a proper scoring rule, which is valid under risk-neutrality, by binarizing it in a way that the agent gets a fixed amount of money if the score is less than a uniform random variable.”
- The Binarized Scoring Rule of Belief Elicitation University of Toronto economics seminar file (Tanjim Hossain and Ryo Okui working paper) · October 2010
“Theoretically, our method formalizes and generalizes the intuition in Roth and Malouf (1979) who suggest paying subjects with tickets for a binary lottery instead of direct monetary payments to induce risk-neutral behavior.”
Reasoning
Textbook excerpts (E2, E5, E9, E10) show that mathematical finance treats binary (digital) and vanilla payoffs as standard, distinct concepts, which supports the first half of the claim. For decision theory, no review or search of the literature is given. Granger and Machina (E1) say Bayesian decision theory already knew about using the full distribution and a decision-maker's cost function, but they do not address binary versus vanilla payoffs. The binarized scoring rule paper (E11, E12) uses binary lotteries in belief elicitation, but the quoted passages do not address the binary-versus-vanilla distinction. The subject's own paper (E3) is not independent. So how far the distinction had reached decision theory remains unsettled.
Cite this claim
Nassim Taleb, “The difference between binary and vanilla payoffs is well known in mathematical finance but does not seem to have been passed on to the decision-theory literature.” (said 2019). Verdict: unresolvable. Reliability API, claim c_30e59a28cdfb, method 0.1.0+28da144b. https://epistacklabs.com/claims/c_30e59a28cdfb
Claims without a verdict
Is a verdict here wrong, or do you speak for Nassim Taleb? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.