Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
82 76 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
61 47 75 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
67 46 85
All types combined: 77 (likely 72 to 82, high confidence), from 84 claims: 63 factual claims, 16 predictions, 5 promises. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Nate Silver's claims
37 claims
“Overall we were quite happy and the forecasts were largely accurate and well-calibrated despite a bad year for the polls.”
Nate Silver, writing on X 8 June 2021 Post by @NateSilver538 on X, 2021-06-08 (X)
The 2020 election forecasts by Nate Silver's team were largely accurate and well-calibrated despite a bad year for the polls.
Mostly true True in substance, wrong in a detail.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judge whether FiveThirtyEight's 2020 presidential, Senate and House forecasts were largely accurate and well-calibrated, given that 2020 polling error was large. True: FiveThirtyEight's own calibration analysis or independent evaluations show that the forecasts called most races correctly (including the presidential winner) and that the stated probabilities matched outcome frequencies reasonably well, with no major systematic overconfidence. Mostly true: they were accurate and broadly calibrated, with a noticeable but limited flaw (e.g. leaning toward Democrats in close races). Mixed: they were accurate on winners but clearly poorly calibrated on margins or probabilities, or evaluations are split. Mostly false: the forecasts were notably miscalibrated or missed many races, and they look acceptable only on a narrow measure. False: evaluations show the forecasts were inaccurate and poorly calibrated.
Evidence
- The Polls and the U.S. Presidential Election in 2020 …. and 2024 MIT DSpace (Barnett & Sarfati, Statistics and Public Policy, Taylor & Francis) · 30 May 2023
“We find that, taken together, the pollsters and FiveThirtyEight did an excellent job in predicting who would win in individual states, even those “tipping point” states where forecasting is more difficult. However, we also find that FiveThirtyEight underestimated Donald Trump’s vote shares by state to a modest but statistically significant extent.”
- Models vs. Markets: Forecasting the 2020 U.S. Election | The Journal of Prediction Markets The Journal of Prediction Markets (Universal Brain / UBPLJ) · 28 December 2023
“Our analysis finds that PredictIt and Five Thirty Eight perform comparably based on traditional metrics such as calibration and accuracy.”
- Who won the 2020 Model Wars? – Mick Bransfield Mick Bransfield (personal blog) · 29 December 2021
“FiveThirtyEight’s lower Brier score is driven by its more cautious predictions in Florida and North Carolina, the two states both models got wrong.”
Reasoning
An independent peer-reviewed analysis (E1) finds that FiveThirtyEight did an excellent job of predicting state winners, including tipping-point states. It also finds that FiveThirtyEight underestimated Trump's state vote shares by a modest but statistically significant amount, which is a limited lean toward Democrats. E2 reports that FiveThirtyEight matched PredictIt on calibration and accuracy, and E4 notes that it called Florida and North Carolina wrong but with cautious probabilities. The evidence covers only the presidential forecasts, not the Senate and House forecasts, and supports accurate, broadly calibrated forecasts with a noticeable but limited flaw.
Cite this claim
Nate Silver, “The 2020 election forecasts by Nate Silver's team were largely accurate and well-calibrated despite a bad year for the polls.” (said 8 June 2021). Verdict: mostly true. Reliability API, claim c_a06055a5b6e2, method 0.1.0+28da144b. https://epistacklabs.com/claims/c_a06055a5b6e2
Claims without a verdict
Is a verdict here wrong, or do you speak for Nate Silver? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.