Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
74 68 79 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
Insufficient record: 1 prediction resolved, 5 needed for a score.
-
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 1 promise resolved, 5 needed for a score.
All types combined: 74 (likely 68 to 79, high confidence), from 97 claims: 97 factual claims. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Steven Pinker's claims
15 claims
“There are studies that show that people, when evaluating data from a hypothetical experiment—if it's politically neutral, like the efficacy of a skin cream—do a decent job of interpreting the numbers.”
Steven Pinker, interviewee 22 March 2018 Steven Pinker Wants Enlightenment Now! (Reason)
Studies show that people evaluating data from a politically neutral hypothetical experiment, such as a skin cream's efficacy, interpret the numbers reasonably well.
Mixed Partly true and partly false in roughly equal measure.
The second judge said mostly true; the verdict shown is the first judge's.
What would count as true or false
Judged as of 2018-03-22. True: published research (for example Kahan et al. 2013/2017, 'Motivated Numeracy') shows that participants, especially numerate ones, interpreted data from a neutral hypothetical skin-cream experiment reasonably well compared with a politically charged version of the same data. Mostly true: such studies exist and support the point, but with a minor qualification, for example that only the more numerate participants did well while many people overall answered wrongly. Mixed: the studies show neutral framing improved interpretation, but overall accuracy was low enough that 'a decent job' is a doubtful description. Mostly false: the cited research exists but the finding is substantially misstated. False: no such studies exist, or they found people interpreted neutral data poorly with no advantage from neutral framing.
Evidence
- Do the math? Only if I agree with it! - The Boston Globe The Boston Globe · 20 October 2013
“With the skin rash question, the results were straightforward. Low-numerate people struggled to come to the correct, data-supported conclusion no matter their political leanings. They were likely to get it right no better than about 38 percent of the time, compared with a success rate of about 65 percent among high-numerate people.”
- Do the math? Only if I agree with it! - The Boston Globe The Boston Globe · 20 October 2013
“In fact, to look at the study’s results from a different perspective, nearly 60 percent of participants overall weren’t able to answer the question correctly, no matter which question they received.”
- Do the math? Only if I agree with it! - The Boston Globe The Boston Globe · 20 October 2013
“They were likely to get the question right about one-third of the time, compared to two-thirds of the time for the group with better math skills.”
- We asked people to do climate change maths. Their answers depended on their politics Australian National Centre for the Public Awareness of Science, ANU (republished from The Conversation) · 19 June 2019
“Participants in our study got the correct mathematical answer 48% of the time.”
Reasoning
The evidence confirms that such a study exists, with a skin-rash treatment version of the question (E1, E3). In that version, low-numerate participants answered correctly only about a third of the time (about 38%), and high-numerate participants about two-thirds of the time (about 65%) (E1, E3). Across the versions of the question, nearly 60% of participants answered incorrectly (E2). So the research supports the point only partly: on the skin-rash question, high-numerate participants answered correctly about 65% of the time but low-numerate participants only about 38% of the time (E1), and nearly 60% of all participants answered incorrectly whichever version they received (E2), so "a decent job" is a doubtful description of people in general.
Cite this claim
Steven Pinker, “Studies show that people evaluating data from a politically neutral hypothetical experiment, such as a skin cream's efficacy, interpret the numbers reasonably well.” (said 22 March 2018). Verdict: mixed. Reliability API, claim c_07a78a3aa171, method 0.1.0+28da144b. https://epistacklabs.com/claims/c_07a78a3aa171
Claims without a verdict
Is a verdict here wrong, or do you speak for Steven Pinker? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.