Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
80 72 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
70 62 78 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 2 promises resolved, 5 needed for a score.
All types combined: 75 (likely 69 to 80, high confidence), from 79 claims: 35 factual claims, 44 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Scott Alexander's claims
35 claims
“This is almost monotonically decreasing. Every day it’s lower than the day before.”
Scott Alexander, ACX author 14 March 2022 Mantic Monday 3/14/22 (Astral Codex Ten)
The probability shown in the chart in the post was almost monotonically decreasing, lower each day than the day before.
Mostly true True in substance, wrong in a detail.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
This claim is about the chart in the March 2022 post. True: the probability in that chart falls, or stays flat, from day to day over the period shown, with at most a few small upticks, so 'almost monotonically decreasing' is a fair description. Mostly true: the trend is clearly downward with a handful of small daily rises. Mixed: the trend is downward overall, but there are several noticeable rises. Mostly false: the series falls overall but swings up and down a lot, so the description overstates how steady it is. False: the series does not fall over the period, or it moves up and down with no clear trend.
Evidence
- Ukraine #4: Prediction Market Movement Modeling Zvi Mowshowitz (thezvi.wordpress.com) · 15 March 2022
“This is a steady climb downwards, and Scott wonders about whether it is a predictable and thus false pattern.”
- Ukraine #4: Prediction Market Movement Modeling Zvi Mowshowitz (thezvi.wordpress.com) · 15 March 2022
“Each day of this makes it more likely Kyiv can hold out longer, reducing chances of a fall each day going forward, so things should reduce faster than the proportion of time expired.”
- Mantic Monday 3/21/22 Astral Codex Ten (Scott Alexander) · 21 March 2022 Own record
“Will Kiev fall to Russian forces by April 2022?: 14% —→ 2%”
Reasoning
E1 is an independent commentator writing the day after the post. It calls the chart Scott discussed "a steady climb downwards", which fits the "almost monotonically decreasing" description. E2 gives a reason the Kyiv-fall probability would keep falling day by day, but it is reasoning, not a description of the chart. E3 comes from the subject and covers a later period, a drop from 14% to 2%. So it fits a continuing decline but does not describe the chart itself. Only one independent source describes the chart, and it does not give day-by-day values, so the verdict is mostly true rather than fully true.
Cite this claim
Scott Alexander, “The probability shown in the chart in the post was almost monotonically decreasing, lower each day than the day before.” (said 14 March 2022). Verdict: mostly true. Reliability API, claim c_5c96ace491c1, method 0.1.0+8bea457b. https://epistacklabs.com/claims/c_5c96ace491c1
Claims without a verdict
Is a verdict here wrong, or do you speak for Scott Alexander? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.