Claims

Factual · Philosophy, ethics and society

“But as soon as it's a political hot button, like concealed weapon laws, then they'll systematically misread the data in the direction that favors the position associated with their coalition.”

Steven Pinker, interviewee · 22 March 2018 · Steven Pinker Wants Enlightenment Now! (Reason)

Studies show that when data concern a political hot-button issue such as concealed weapon laws, people systematically misread it in the direction favoring their coalition's position.

Mostly true True in substance, wrong in a detail.

The second judge said mixed; the verdict shown is the first judge's.

What would count as true or false

Judge as of 2018-03-22. True: published studies (for example Kahan et al., 'Motivated Numeracy and Enlightened Self-Government', which used concealed-carry gun-control data) show that people interpret the same numerical data less accurately when the data bear on a politically charged issue such as concealed-weapon laws, and that the errors lean systematically toward their political group's position, while accuracy is higher on a neutral topic. Mostly true: such studies exist and support the finding, but a minor detail is off (for example the exact topic or the size of the effect). Mixed: some studies support it, but replications or other work find the effect weak, inconsistent or limited. Mostly false: the evidence is a single early or narrow finding, presented as a general, established pattern, and later replications largely failed. False: studies show no such systematic partisan misreading.

Evidence

1. Motivated numeracy and enlightened self-government | Behavioural Public Policy | Cambridge Core Cambridge University Press (Cambridge Core) · 4 May 2017

   “Also as expected, subjects’ responses became politically polarized – and even less accurate – when the same data were presented as results from the study of a gun control ban.”

2. Motivated numeracy and enlightened self-government | Behavioural Public Policy | Cambridge Core Cambridge University Press (Cambridge Core) · 4 May 2017

   “As expected, subjects highest in numeracy – a measure of the ability and disposition to make use of quantitative information – did substantially better than less numerate ones when the data were presented as results from a study of a new skin rash treatment.”

3. Judgment and Decision Making, Vol. 15, No. 2, March 2020, pp. 203–213 Judgment and Decision Making (Cambridge Core) · March 2020

   “However, when the problems presented a politicized topic (e.g., concealed carry handgun laws and their effects on crime), numeracy did not straightforwardly increase correct responding.”

4. Judgment and Decision Making, Vol. 15, No. 2, March 2020, pp. 203–213 Judgment and Decision Making (Cambridge Core) · March 2020

   “Although one recent conference presentation reported being unable to reproduce these findings (Ballarini & Sloman, 2017), Kahan and Peters (2017) argued that it had insufficient statistical power and differed from the original study in other relevant ways.”

Reasoning

Kahan et al.'s 'Motivated Numeracy' study found that answers became politically polarized and less accurate when the same data were framed as a gun-control ban study [E1]. On a neutral skin-rash framing, the most numerate subjects did substantially better [E2]. This matches the pattern the claim describes. A later paper also found that numeracy did not straightforwardly improve accuracy on a concealed-carry problem [E3], but it notes a 2017 conference presentation that could not reproduce the finding, which the original authors disputed [E4]. That points to some unsettled replication evidence rather than a fully established effect.

Cite this claim

Steven Pinker, “Studies show that when data concern a political hot-button issue such as concealed weapon laws, people systematically misread it in the direction favoring their coalition's position.” (said 22 March 2018). Verdict: mostly true. Reliability API, claim c_e595b7a7eadb, method 0.1.0+28da144b. https://epistacklabs.com/claims/c_e595b7a7eadb