Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
82 76 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
61 47 75 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
67 46 85
All types combined: 77 (likely 72 to 82, high confidence), from 84 claims: 63 factual claims, 16 predictions, 5 promises. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Nate Silver's claims
63 claims
“But this is based on objective analysis of polling in presidential elections dating back to 1972.”
Nate Silver, FiveThirtyEight founder, interview answer 11 October 2016 Nate Silver On Why FiveThirtyEight Forecasts A Clinton Win (WBEZ Chicago)
FiveThirtyEight's forecast of Clinton's chances is based on objective analysis of polling in presidential elections dating back to 1972.
Resolved true The record shows it happened or was so.
The second judge would not resolve this claim; the verdict shown is the first judge's.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judged as of 2016-10-11. 'Objective' is read as systematic and rules-based (a quantitative model, not subjective judgment). True: FiveThirtyEight's published 2016 methodology shows its presidential forecast was built or calibrated with historical presidential polling data going back to 1972, applied by a fixed quantitative procedure. Mostly true: the model is quantitative and uses historical polling, but the start year differs slightly (e.g. a few cycles) or 1972 applies only to some components. Mixed: historical polling back to about 1972 is used for only a minor part of the model, or the model involves significant discretionary judgment calls. Mostly false: the model uses historical data but from a substantially shorter period, or it rests mainly on non-polling inputs. False: the methodology shows no such historical polling calibration.
Evidence
- [PDF] Deciphering the Signal and the Noise - Ben Tanen Ben Tanen (Tufts University term paper, Engineering Management 52) · 10 May 2017
“FiveThirtyEight’s 2016 Election Forecast primarily relied on outside polling data.”
- [PDF] Deciphering the Signal and the Noise - Ben Tanen Ben Tanen (Tufts University term paper, Engineering Management 52) · 10 May 2017
“average (over the prior 50 years), additional percentage points were assigned to the candidate”
- dsbook-part-2/inference/hierarchical-models.qmd at main · rafalab/dsbook-part-2 GitHub (rafalab, Rafael Irizarry's Advanced Data Science book source)
“In this section, we illustrate the use of hierarchical models by describing a simplified version of the approach used by FiveThirtyEight to forecast the 2016 election.”
- Why No One Knew that Hillary Clinton Would Lose the 2016 Election American Journal of Political Science Review (Buresh & Pavone), via Touro Scholar · 19 November 2018
“Polls-plus – It combines polls with an economic index.”
- Deciphering the Signal and the Noise - Paper Tufts University (student term paper by Benjamin Tanen) · 10 May 2017
“Once the model calculated a weighted polling average for each state, FiveThirtyEight's Election Forecast then made poll-specific adjustments to these averages (Silver 2016b).”
- Deciphering the Signal and the Noise - Paper Tufts University (student term paper by Benjamin Tanen) · 10 May 2017
“Historically, Republicans generally have a slight advantage (between one and two percentage points) in polls of likely voters. The Election Forecast model took this historical advantage into account in its calculation of averages.”
- Why No One Knew that Hillary Clinton Would Lose the 2016 Election Touro University Worldwide (Touro Scholar), published in American Journal of Political Science Review · 19 November 2019
“All three versions of the predictive model collected weight and averaged poll numbers, adjusted the poll numbers, combined the poll numbers with demographic and economic data, and accounted for uncertainty and simulated the election thousands of times.”
- dsbook-part-2/inference/hierarchical-models.qmd at main · rafalab/dsbook-part-2 GitHub (textbook source by rafalab, Advanced Data Science book)
“We don't show the data here, but if we took the average of polls for each past election and compared it to the actual election night result, we would observe a difference with a standard deviation of between 2% and 4%.”
- Donald Trump has a 20 percent chance of becoming president ABC7 (reprinting FiveThirtyEight) · 29 June 2016 Own record
“Our models are built from data since 1972, so the probabilities we list account for elections such as 1980, 1988 and 1992, when the polls swung fairly wildly, along with others, such as 2004 and 2012, where the polls were quite stable.”
- Donald Trump has a 20 percent chance of becoming president ABC7 (reprinting FiveThirtyEight) · 29 June 2016 Own record
“It doesn't use fundraising or political experience as a factor in making its forecasts - instead, it just uses polls and demographic data.”
- Donald Trump has a 20 percent chance of becoming president ABC7 (reprinting FiveThirtyEight) · 29 June 2016 Own record
“The choice of assumptions can matter quite a bit when building models, especially in cases such as presidential elections where the historical sample sizes are small.”
- Five Thirty Eight Model: Trump Wins Georgia, But Clinton Wins Nationally in a Landslide Georgia Political Science blog (georgiapol.com) · 30 June 2016
“The final model, polls plus, uses economic data and past voter behavior to make assumptions about how the election will turn out.”
Reasoning
FiveThirtyEight's own published 2016 forecast write-up (E9) says its models are "built from data since 1972" and that its probabilities account for past elections with volatile and stable polls. It also says the forecast uses polls and demographic data rather than factors such as fundraising (E10). Independent accounts (E1, E5, E6, E7) describe the forecast as mainly a polling model, with adjustments to poll averages, historical corrections and simulation, which indicates a quantitative procedure. Only FiveThirtyEight's own write-up gives the 1972 start year, and the same write-up notes that modelling assumptions matter (E11). Even so, the evidence meets the criteria for true.
Cite this claim
Nate Silver, “FiveThirtyEight's forecast of Clinton's chances is based on objective analysis of polling in presidential elections dating back to 1972.” (said 11 October 2016). Verdict: resolved true. Reliability API, claim c_1035abf13f7f, method 0.1.0+28da144b. https://epistacklabs.com/claims/c_1035abf13f7f
Claims without a verdict
Is a verdict here wrong, or do you speak for Nate Silver? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.