ClaimsNate Silver

“This mostly makes a difference for very low-probability events.”

Nate Silver, FiveThirtyEight editor-in-chief, column author 2016 Election Update: Why Our Model Is More Bullish Than Others On Trump (FiveThirtyEight) archived copy

Factual Science and research methods

The t-distribution in FiveThirtyEight's model mostly makes a difference for very low-probability events.

Mostly true True in substance, wrong in a detail.

Factual: Statements about the present or the past, checked against the record: did they hold up?

The second judge would not resolve this claim; the verdict shown is the first judge's.

Said for an organisation they lead or work for; it counts on both records.

What would count as true or false

Judged as of 2016. The claim is that using a fat-tailed t-distribution rather than a normal distribution in FiveThirtyEight's election model changed results mainly for very low-probability outcomes, i.e. in the tails. True: FiveThirtyEight's methodology documents, together with standard statistical properties, show the t-distribution's effect is concentrated in tail probabilities (e.g. a candidate with a small chance of winning a state) and is small for outcomes near the middle. Mostly true: the effect is concentrated in the tails but also has some non-trivial effect on moderate probabilities. Mixed: the effect is spread fairly evenly between tail and moderate probabilities. Mostly false: the t-distribution materially changed moderate probabilities as well. False: the model did not use a t-distribution, or the effect was mainly on moderate probabilities.

Set before the research began.

Evidence (4)

  1. E1 What’s Wrong With 538? HuffPost · 7 November 2016

    “I am not questioning any specific assumption that 538 makes, from their state-by-state correlations to their use of a t-distribution to create “fat tails” in their probability distributions (i.e., higher likelihood of otherwise obscure events).”
  2. E2 What’s Wrong With 538? HuffPost · 7 November 2016

    “Perhaps Nate Silver can say that he’s gotten all of these improbable outcomes because of his use of the t-distribution rather than a normal distribution to accentuate the tails of the model.”
  3. E3 Accurate Prediction of Electoral Outcomes arXiv (academic preprint by outside authors)

    “If we study the histogram in Figure 24, it is not clear that fat-tailed distributions are relevant here, unless we believe there are non-trivial probabilities of a landslide.”
  4. E4 Accurate Prediction of Electoral Outcomes arXiv (academic preprint by outside authors)

    “The probabilities provided by the Student T regression as well as the Hierarchical Model were very similar to the CAPM Model.”

Reasoning

An independent 2016 HuffPost piece confirms that FiveThirtyEight used a t-distribution to create "fat tails" in its probability distributions, which it describes as a "higher likelihood of otherwise obscure events" [E1]. The same piece describes the t-distribution as a way to "accentuate the tails of the model" [E2]. That matches the claim that the effect falls mainly on low-probability outcomes. No FiveThirtyEight methodology document is in the evidence, and nothing measures how much the choice changed moderate probabilities, so the claim cannot be fully confirmed. The arXiv preprint [E3, E4] studies a different model, so it is not used.

Cite this claim

Nate Silver, “The t-distribution in FiveThirtyEight's model mostly makes a difference for very low-probability events.” (said 2016). Verdict: mostly true. Reliability API, claim c_4c6e5be817ea, method 0.1.0+28da144b, dataset release v1.1.2. https://epistacklabs.com/claims/c_4c6e5be817ea

This claim is in dataset release v1.1.2, with the same verdict.

Reply to this claim

For Nate Silver, or someone speaking for them: a correction or counter-evidence. Replies are read before anything is shown; an approved reply shows your name, role, date, statement and link. Your email address is never shown and is deleted 30 days after the decision (privacy).

More factual claims by Nate Silver

  1. Mostly true Factual 9 October 2024
    “The previous Quinnipiac polls of these states had been quite a bit better for Harris.”

    The previous Quinnipiac polls of Michigan, Wisconsin and Pennsylvania were quite a bit better for Harris than the latest ones.

  2. Resolved true Factual 9 October 2024
    “Although John McCain had periods of strong polling in 2008, especially after the Republican Convention, in the end Barack Obama won in a landslide that didn’t justify Democrats’ constant anxieties.”

    John McCain had periods of strong polling in 2008, especially after the Republican Convention, but Barack Obama won in a landslide.

  3. Resolved true Factual 9 October 2024
    “The recognition that the election is only four weeks away.”

    As of early October 2024, the US presidential election is about four weeks away.

  4. Resolved true Factual 9 October 2024
    “And yet, there are dozens of polls released every week.”

    Dozens of election polls are released every week.

  5. Mostly true Factual 8 October 2024
    “This property is even more profound in baseball, where single-run leads are relatively durable because runs tend to come in bunches and the default outcome is not to score at all.”

    In baseball, single-run leads are relatively durable because runs tend to come in bunches and the default outcome is not to score at all.

All of them

Verdicts are made automatically; how verdicts are made.