The lack of understanding of how AI systems work is essentially unprecedented in the history of technology.

The claim is hedged ('essentially'). It says that how poorly the internal workings of modern AI systems are understood has essentially no precedent among major technologies. TRUE: expert or historical analyses support that widely deployed technologies have generally been built on a reasonably good mechanistic understanding, and that no comparable case exists of a major deployed technology whose workings were as opaque to its builders. MOSTLY TRUE: the claim broadly holds, but there are partial historical parallels (for example early pharmaceuticals such as aspirin, metallurgy, or early steam engines before thermodynamics) that make 'unprecedented' slightly too strong. MIXED: significant precedents exist that are arguably comparable, and expert opinion is divided. MOSTLY FALSE: clear precedents exist of important technologies deployed with little understanding of how they work, so the claim exaggerates a real gap in understanding. FALSE: it was common historically to deploy technologies whose mechanisms were similarly poorly understood, and AI is not distinctive in this respect.

Set before the research began.

1. E1 The Virtues of Interpretable Medical AI Cambridge Quarterly of Healthcare Ethics (authors: Joshua Hatherley, Robert Sparrow, Mark Howard, Monash University) · 2024

   “Other authors have also highlighted acetaminophen and penicillin, which “were in widespread use for decades before their mechanism of action was understood,” along with selective serotonin reuptake inhibitors, whose underlying causal mechanism is still unclear.”

2. E2 The Virtues of Interpretable Medical AI Cambridge Quarterly of Healthcare Ethics (authors: Joshua Hatherley, Robert Sparrow, Mark Howard, Monash University) · 2024

   “For example, electroconvulsive therapy is highly effective for severe depression, but we have no idea how it works. Likewise, there are many drugs that seem to work even though no one can explain how.”

3. E3 The Virtues of Interpretable Medical AI Cambridge Quarterly of Healthcare Ethics (authors: Joshua Hatherley, Robert Sparrow, Mark Howard, Monash University) · 2024

   “A problem, however, is that the inner workings of these systems have often proven thoroughly resistant to understanding, explanation, or justification, not only to end users (e.g., doctors, clinicians, and nurses), but even to the designers of these systems themselves.”

A medical ethics journal article (E3) says the inner workings of these AI systems have often resisted understanding, even by the people who designed them. That supports the premise that AI is unusually opaque. The same article also records partial historical parallels: acetaminophen and penicillin were widely used for decades before anyone understood how they worked, the mechanism of SSRIs is still unclear (E1), and electroconvulsive therapy and many drugs work in ways no one can explain (E2). The resolution criteria name early pharmaceuticals as exactly the kind of partial parallel that makes "unprecedented" slightly too strong, so the hedged claim broadly holds but is somewhat overstated.