Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
67 61 73 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
60 44 76 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 0 promises resolved, 5 needed for a score.
All types combined: 66 (likely 61 to 72, high confidence), from 105 claims: 94 factual claims, 11 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Marc Andreessen's claims
15 claims
“We know one-to-one tutoring can reliably increase education outcomes by two standard deviations (the Bloom two-sigma effect)”
Marc Andreessen, a16z co-founder, essay author 18 April 2020 It's Time to Build (a16z)
One-to-one tutoring can reliably increase education outcomes by two standard deviations (the Bloom two-sigma effect).
Mostly false False in substance, right in a detail.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judged by the state of research as of 2020-04-18. True: the research literature shows that one-to-one tutoring reliably (consistently, across studies and settings) produces gains of about two standard deviations. Mostly true: replicated gains are close to 2 SD (say 1.5 SD or more) in most rigorous studies. Mixed: Bloom's 1984 studies did report about 2 SD under specific conditions (mastery learning, short durations, particular tests), but replication is inconsistent. Mostly false: Bloom did report the 2-sigma figure, but meta-analyses and replications available by 2020 show typical tutoring effects that are substantially smaller (for example about 0.3 to 0.8 SD), so calling it 'reliable' at 2 SD overstates a narrow early finding. False: no study ever reported a 2 SD tutoring effect, or the evidence shows tutoring has no positive effect.
Evidence
- The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring Educational Researcher (Benjamin S. Bloom, 1984) · 1984
“However, the most striking of the findings is that under the best learning conditions we can devise (tutoring), the average student is 2 sigma above the average control student taught under conventional group methods of instruction.”
- The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems. ERIC (abstract of Kurt VanLehn, Educational Psychologist 46(4)) · 2011
“In particular, when compared to No tutoring, the effect sizes of answer-based tutoring systems, intelligent tutoring systems, and adult human tutors are believed to be d = 0.3, 1.0, and 2.0 respectively. This review did not confirm these beliefs. Instead, it found that the effect size of human tutoring was much lower: d = 0.79.”
- Two-Sigma Tutoring: Separating Science Fiction from Science Fact Education Next · 7 March 2024
“More recently, a 2020 meta-analysis of randomized studies by Andre Nickow, Philip Oreopoulos, and Vincent Quan found that the average effect of tutoring was 0.37 standard deviations, or 14 percentile points”
Reasoning
Bloom's 1984 paper reports that tutored students averaged 2 sigma above students taught in conventional groups (E2), so the figure itself does exist. VanLehn's 2011 review, published well before 2020, tested the common belief that human tutoring has an effect size of 2.0 and did not confirm it. It found d = 0.79 instead (E3). E4 adds a 2020 meta-analysis of randomized studies with an average tutoring effect of 0.37 SD. Because that meta-analysis may be from later in 2020 than the claim, the verdict mainly rests on E3. The evidence shows typical tutoring effects well below 2 SD, so calling the 2-sigma effect "reliable" overstates an early, narrow finding.
Cite this claim
Marc Andreessen, “One-to-one tutoring can reliably increase education outcomes by two standard deviations (the Bloom two-sigma effect).” (said 18 April 2020). Verdict: mostly false. Reliability API, claim c_d3ffc85cced0, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_d3ffc85cced0
Claims without a verdict
Is a verdict here wrong, or do you speak for Marc Andreessen? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.