The t-distribution in FiveThirtyEight's model mostly makes a difference for very low-probability events.
Mostly true True in substance, wrong in a detail.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judged as of 2016. The claim is that using a fat-tailed t-distribution rather than a normal distribution in FiveThirtyEight's election model changed results mainly for very low-probability outcomes, i.e. in the tails. True: FiveThirtyEight's methodology documents, together with standard statistical properties, show the t-distribution's effect is concentrated in tail probabilities (e.g. a candidate with a small chance of winning a state) and is small for outcomes near the middle. Mostly true: the effect is concentrated in the tails but also has some non-trivial effect on moderate probabilities. Mixed: the effect is spread fairly evenly between tail and moderate probabilities. Mostly false: the t-distribution materially changed moderate probabilities as well. False: the model did not use a t-distribution, or the effect was mainly on moderate probabilities.
Set before the research began.
Evidence (4)
-
E1 What’s Wrong With 538? HuffPost · 7 November 2016
“I am not questioning any specific assumption that 538 makes, from their state-by-state correlations to their use of a t-distribution to create “fat tails” in their probability distributions (i.e., higher likelihood of otherwise obscure events).”
-
E2 What’s Wrong With 538? HuffPost · 7 November 2016
“Perhaps Nate Silver can say that he’s gotten all of these improbable outcomes because of his use of the t-distribution rather than a normal distribution to accentuate the tails of the model.”
-
E3 Accurate Prediction of Electoral Outcomes arXiv (academic preprint by outside authors)
“If we study the histogram in Figure 24, it is not clear that fat-tailed distributions are relevant here, unless we believe there are non-trivial probabilities of a landslide.”
-
E4 Accurate Prediction of Electoral Outcomes arXiv (academic preprint by outside authors)
“The probabilities provided by the Student T regression as well as the Hierarchical Model were very similar to the CAPM Model.”
Reasoning
An independent 2016 HuffPost piece confirms that FiveThirtyEight used a t-distribution to create "fat tails" in its probability distributions, which it describes as a "higher likelihood of otherwise obscure events" [E1]. The same piece describes the t-distribution as a way to "accentuate the tails of the model" [E2]. That matches the claim that the effect falls mainly on low-probability outcomes. No FiveThirtyEight methodology document is in the evidence, and nothing measures how much the choice changed moderate probabilities, so the claim cannot be fully confirmed. The arXiv preprint [E3, E4] studies a different model, so it is not used.
Cite this claim
Nate Silver, “The t-distribution in FiveThirtyEight's model mostly makes a difference for very low-probability events.” (said 2016). Verdict: mostly true. Reliability API, claim c_4c6e5be817ea, method 0.1.0+28da144b, dataset release v1.1.2. https://epistacklabs.com/claims/c_4c6e5be817ea
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Nate Silver
-
Mostly true
“The previous Quinnipiac polls of these states had been quite a bit better for Harris.”
-
Resolved true
“Although John McCain had periods of strong polling in 2008, especially after the Republican Convention, in the end Barack Obama won in a landslide that didn’t justify Democrats’ constant anxieties.”
-
Resolved true
“The recognition that the election is only four weeks away.”
As of early October 2024, the US presidential election is about four weeks away.
-
Resolved true
“And yet, there are dozens of polls released every week.”
-
Mostly true
“This property is even more profound in baseball, where single-run leads are relatively durable because runs tend to come in bunches and the default outcome is not to score at all.”
Verdicts are made automatically; how verdicts are made.