The obvious experimental manipulations of heuristics-and-biases studies have already been tried, and the results were found to be robust.
Unresolvable The public record could not settle it.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
True: as of 2008, the heuristics-and-biases literature shows that the obvious alternative explanations and manipulations had been tested and the main effects were found to replicate robustly. Such manipulations include incentives, rewording, expertise, and debiasing instructions. Examples are anchoring, conjunction fallacy, overconfidence and planning fallacy studies, as reviewed in Gilovich, Griffin and Kahneman (2002) or similar. Mostly true: most major effects were robust to such manipulations as of 2008, but some had known boundary conditions. For example, frequency formats reduced the conjunction fallacy and base-rate neglect. Mixed: robustness as of 2008 was substantially contested, with roughly as many effects shown to weaken under obvious manipulations as remained robust. Mostly false: the literature as of 2008 showed many key effects shrank or disappeared under obvious manipulations, and only a minority were robust. False: the obvious manipulations had not been tried, or they generally eliminated the effects. Later replication-crisis findings count only if they formally correct the specific findings that the claim relies on.
Set before the research began.
Evidence (4)
-
E1 ramifications Not shown on page (book edited by Gilovich, Griffin and Kahneman, Heuristics and Biases; hosted copy at bear.warrington.ufl.edu)
“made rationality violations disappear purely by raising incentives”
-
E2 ramifications Not shown on page (book edited by Gilovich, Griffin and Kahneman, Heuristics and Biases; hosted copy at bear.warrington.ufl.edu)
“it is equally clear that the main biases uncovered in this research tradition”
-
E3 Chapter 20 THE COGNITIVE ILLUSION CONTROVERSY Not shown on page (chapter by Ralph Hertwig and Andreas Ortmann)
“disappear by representing statistical information differently than it typically had”
-
E4 Chapter 20 THE COGNITIVE ILLUSION CONTROVERSY Not shown on page (chapter by Ralph Hertwig and Andreas Ortmann)
“Kleinbölting (1991) argued that this is why people appear overconfident in the”
Reasoning
The evidence shows that the literature discussed incentives (E1), the main biases (E2), changes to how statistical information is represented (E3) and explanations for overconfidence (E4). None of these excerpts says whether the main effects stayed robust under the obvious manipulations, so the evidence does not settle the claim under the criteria.
Cite this claim
Eliezer Yudkowsky, “The obvious experimental manipulations of heuristics-and-biases studies have already been tried, and the results were found to be robust.” (said 2008). Verdict: unresolvable. Reliability API, claim c_79614252619b, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_79614252619b
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Eliezer Yudkowsky
-
Mostly true
“And that in turn provided a lot of indirect support for the atomic theory.”
Conservation of mass provided a lot of indirect support for atomic theory.
-
Mostly true
“The conservation of mass was one of the first hints that things were being rearranged rather than created or destroyed.”
-
Resolved true
“The way that if you burn something inside a sealed glass vessel, it didn't change weight.”
Burning a substance inside a sealed glass vessel does not change the total weight.
-
Mostly true
“that ignorance is a mental rather than physical property.”
-
Mixed
“General intelligence is a between-species difference, a complex adaptation, and a human universal found in all known cultures.”
Verdicts are made automatically; how verdicts are made.