Almost nothing is 40% causative of anything, apart from a very small number of exceptions.
Mostly true True in substance, wrong in a detail.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
The claim is a general statement about effect sizes: very few single factors account for about 40% or more of the causation (or variance) of any outcome. True: reviews or meta-analyses of effect sizes across fields (social science, medicine, genetics) show that single factors explaining about 40% or more of an outcome are rare exceptions. Mostly true: such effects are uncommon but somewhat more frequent than 'a very small number', or this holds in the fields the claim plausibly refers to but not universally. Mixed: the evidence is split, with large single-factor effects common in some major domains and rare in others. Mostly false: the claim is stated as established but the literature cannot settle it, or it overstates a real tendency for effects to be small. False: the literature shows single-factor effects of about 40% or more are common across many domains.
Set before the research began.
Evidence (11)
-
E1 Evaluating Effect Size in Psychological Research: Sense and Nonsense Association for Psychological Science (Advances in Methods and Practices in Psychological Science) · 1 June 2019
“A very large effect size ( r = .40 or greater) in the context of psychological research is likely to be a gross overestimate that will rarely be found in a large sample or in a replication.”
-
E2 Empirical effect size comparison studies (2020 overview) Emil Kirkegaard (personal blog compilation) · 4 March 2020
“The analysis reveals that the 25th, 50th, and 75th percentiles corresponded to correlation coefficients values of 0.12, 0.25, and 0.42 and to Hedges’ g values of 0.15, 0.38, and 0.69, respectively.”
-
E3 Is psychological science progressing? Explained variance in PsycINFO articles during the period 1956 to 2022 PubMed Central (NCBI); journal name not shown in fetched text · 21 December 2022
“While the average ratio is lower, there are enough cases left where researchers can happily claim to have found a large effect ranging around 40%.”
-
E4 Effect Size Magnification: No Variable Is as Important as the One You’re Thinking About—While You’re Thinking About It SAGE Publications (Current Directions in Psychological Science) · 11 October 2024
“In genetics these are the exception, not the rule (O’Connor, 2021), and we expect fewer to exist in psychology.”
-
E5 Empirical effect size comparison studies (2020 overview) Emil Kirkegaard (personal blog compiling published studies) · 4 March 2020
“The analysis reveals that the 25th, 50th, and 75th percentiles corresponded to correlation coefficients values of 0.12, 0.25, and 0.42 and to Hedges’ g values of 0.15, 0.38, and 0.69, respectively.”
-
E6 Empirical effect size comparison studies (2020 overview) Emil Kirkegaard (personal blog compiling published studies) · 4 March 2020
“Based on the results, it is suggested that Cohen’s correlation guidelines are too exigent, as <3% of correlations in the literature were found to be as large as r = 0.50.”
-
E7 Is psychological science progressing? Explained variance in PsycINFO articles during the period 1956 to 2022 PubMed Central (PMC); journal name not shown in fetched text · 21 December 2022
“For all 1,565 articles, the mean amount of (maximum) explained variance was 42.8 percent with 95 percent confidence interval from 41.7 to 43.9.”
-
E8 Recalibrating expectations about effect size: A multi-method survey of effect sizes in the ABCD study PMC (National Library of Medicine), authors of the ABCD study analysis · 23 September 2021
“In aim one analyses, the median in-sample effect size was .03, and values at the first and third quartiles were .01 and .07.”
-
E9 Recalibrating expectations about effect size: A multi-method survey of effect sizes in the ABCD study PMC (National Library of Medicine), authors of the ABCD study analysis · 23 September 2021
“For example, amongst false discovery rate corrected, between-instrument, mixed effect models (“real-world” analysis), the 90th percentile of effect size was a correlation of .18 and the 95th percentile was .29.”
-
E10 Recalibrating expectations about effect size: A multi-method survey of effect sizes in the ABCD study PMC (National Library of Medicine), authors of the ABCD study analysis · 23 September 2021
“One recent meta-analysis from six prominent psychology journals found that the average published effect size was r = .19 and that the 90th percentile of effect sizes was only r = .41 [7].”
-
E11 One Hundred Years of Social Psychology Quantitatively Described SAGE Publications / Review of General Psychology (APA Division 1) · December 2003
“Analyses reveal that social psychological effects typically yield a value of *r* equal to.21 and that, in the typical research literature, effects vary from study to study in ways that produce a standard deviation in *r* of.15.”
Reasoning
E1 calls a correlation of r = .40 or more very large in psychology and says it is rarely found in large samples or replications. E7 reports that under 3% of literature correlations reach r = .50. E10 and E11 put the upper percentiles of effect sizes at modest levels: a 95th percentile of r = .29 in one dataset and a 90th percentile of r = .41 in another. E5 says large single-variable effects in genetics are the exception, not the rule. Against this, E2/E6 report a 75th-percentile correlation of .42 in one compilation, and E3 notes that some studies still claim large effects of around 40%. The evidence therefore supports the claim mainly for psychology and genetics, not across all fields, which fits mostly true.
Cite this claim
Marc Andreessen, “Almost nothing is 40% causative of anything, apart from a very small number of exceptions.” (said 2 September 2023). Verdict: mostly true. Reliability API, claim c_a4e9b737e802, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_a4e9b737e802
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Marc Andreessen
-
Mixed
“typically in the literature, it's like a 0.4, 0.5 correlation.”
-
Mostly false
“We know one-to-one tutoring can reliably increase education outcomes by two standard deviations (the Bloom two-sigma effect)”
-
Mixed
“It just can’t have been AI.”
-
Mixed
“Basically, within less than 12 months, four or five Chinese companies have caught up to the industry front - line.”
-
Mostly true
“Here's the key: What's happening to the price of AI? It's dropping faster than Moore's Law.”
Verdicts are made automatically; how verdicts are made.