The Brier score is favored because it does not allow arbitrage and requires perfect probabilistic calibration.
Unresolvable The public record could not settle it.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judged against the forecasting and statistics literature as of 2019. True: authoritative sources say the Brier score is a strictly proper scoring rule, meaning it cannot be gamed or arbitraged by misreporting probabilities, and that it rewards calibration, and that these properties are cited as reasons it is widely used. Mostly true: the properties are accurate but slightly mis-stated. For example, the score rewards calibration together with resolution rather than 'requiring perfect calibration', or 'no arbitrage' is a loose paraphrase of properness. Mixed: one of the two stated reasons is accurate and the other is materially wrong or not a recognised reason the score is favoured. Mostly false: both reasons are distorted, for example because properness is real but the 'requires perfect calibration' part is wrong and properness is not why it is favoured. False: the Brier score is shown not to be proper and not to reward calibration.
Set before the research began.
Evidence (4)
-
E1 Strictly Proper Scoring Rules, Prediction, and Estimation Journal of the American Statistical Association (review article by Tilmann Gneiting and Adrian E. Raftery, University of Washington) · March 2007
“Thiswell-knownscoringrulewasproposedbyBrier(1950).”
-
E2 Strictly Proper Scoring Rules, Prediction, and Estimation Journal of the American Statistical Association (review article by Tilmann Gneiting and Adrian E. Raftery, University of Washington) · March 2007
“ThescoringrulesinExamples1–3arestrictlyproper.”
-
E3 Strictly Proper Scoring Rules, Prediction, and Estimation Journal of the American Statistical Association (review article by Tilmann Gneiting and Adrian E. Raftery, University of Washington) · March 2007
“Intermsofelicitation,theroleofscoringrulesistoencouragetheassessortomakecarefulassessmentsandtobehonest”
-
E4 Strictly Proper Scoring Rules, Prediction, and Estimation Journal of the American Statistical Association (review article by Tilmann Gneiting and Adrian E. Raftery, University of Washington) · March 2007
“wecontendedthatthegoalofprobabilisticforecastingistomaximizethesharpnessofthepredictivedistributionssubjecttocalibration.”
Reasoning
The evidence says a well-known scoring rule was proposed by Brier (E1) and that the rules in the paper's Examples 1 to 3 are strictly proper (E2), but no item states that the Brier score is one of those examples. E3 and E4 speak about scoring rules and calibration in general, not about the Brier score, so the evidence does not establish the properties the criteria require.
Cite this claim
Nassim Taleb, “The Brier score is favored because it does not allow arbitrage and requires perfect probabilistic calibration.” (said 2019). Verdict: unresolvable. Reliability API, claim c_71a49c11936c, method 0.1.0+28da144b, dataset release v1.1.2. https://epistacklabs.com/claims/c_71a49c11936c
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Nassim Taleb
-
Mixed
“entities grow in a convex way, then slow as they saturate — growth may be unbounded, but remains sub-logarithmic.”
-
Mixed
“the unavoidable conclusion is that cardio problems are at least half genetic.”
-
Mixed
“scientists that come up with new ideas & frameworks are more often the experienced ones, not the young as we tend to believe.”
-
Mostly true
“And if one does not hedge the other, then being "good at”
-
Mostly true
“A sum of such bets becomes rapidly”
Verdicts are made automatically; how verdicts are made.