Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
80 72 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
70 62 78 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 2 promises resolved, 5 needed for a score.
All types combined: 75 (likely 69 to 80, high confidence), from 79 claims: 35 factual claims, 44 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Scott Alexander's claims
26 claims
“Early in the war, prediction markets didn’t like Ukraine’s odds; now they’re much more sanguine.”
Scott Alexander, ACX author 14 March 2022 Mantic Monday 3/14/22 (Astral Codex Ten)
Early in the Russia-Ukraine war prediction markets gave Ukraine poor odds, and by March 2022 they were much more optimistic about Ukraine.
Mostly true True in substance, wrong in a detail.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
TRUE: prediction markets or forecasting platforms (for example Polymarket, Metaculus, Manifold, Good Judgment, PredictIt) show that in the first days of the invasion (around 24–28 February 2022) they gave Ukraine poor odds (for example a high probability that Kyiv would fall or Zelensky would be ousted), and that by about 14 March 2022 the same or comparable markets had moved clearly in Ukraine's favour. MOSTLY TRUE: the shift happened in that direction but was more moderate than 'much more sanguine', or showed up on only some major markets. MIXED: markets disagreed, some moving markedly and others not. MOSTLY FALSE: early odds were not clearly poor for Ukraine, or there was only a slight improvement. FALSE: market odds for Ukraine did not improve, or got worse, between the start of the war and mid-March 2022.
Evidence
- Ukraine Post #1: Prediction Markets Zvi Mowshowitz (thezvi.wordpress.com) · 28 February 2022
“Will Kyiv fall by April 1? 69%.”
- Ukraine #4: Prediction Market Movement Modeling Zvi Mowshowitz (thezvi.wordpress.com) · 15 March 2022
“Note that the real money version of this is trading lower at 9%.”
- Ukraine #4: Prediction Market Movement Modeling Zvi Mowshowitz (thezvi.wordpress.com) · 15 March 2022
“This is a steady climb downwards, and Scott wonders about whether it is a predictable and thus false pattern.”
- Ukraine #4: Prediction Market Movement Modeling Zvi Mowshowitz (thezvi.wordpress.com) · 15 March 2022
“Back at the start of this market, Kharkiv was 80% to fall, now it’s 72%?”
Reasoning
E1 (28 Feb 2022) puts a forecast question on Kyiv falling by April 1 at 69%, so early forecasts gave Ukraine poor odds. By mid-March, E4 shows a Kharkiv market down from 80% to 72% likely to fall, and E3 describes a "steady climb downwards" that Scott was asking about. Both point toward Ukraine, but the only labelled figures (E4) show a modest move. E2's 9% figure isn't tied to a specific question in the quote, so it can't confirm a big drop in the Kyiv odds. The evidence supports the direction of the claim but doesn't clearly show "much more sanguine" across major markets.
Cite this claim
Scott Alexander, “Early in the Russia-Ukraine war prediction markets gave Ukraine poor odds, and by March 2022 they were much more optimistic about Ukraine.” (said 14 March 2022). Verdict: mostly true. Reliability API, claim c_f898139cebcf, method 0.1.0+8bea457b. https://epistacklabs.com/claims/c_f898139cebcf
Claims without a verdict
Is a verdict here wrong, or do you speak for Scott Alexander? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.