In open-ended tasks such as driving, humans outperform AI systems.

Judge as of 2021-11-16. True: no AI system then matched or beat human performance on open-ended tasks such as general driving across unrestricted conditions. For driving, this means autonomous vehicles still needed safety drivers or remote help, or ran only in geofenced, limited areas, with no credible evidence of better-than-human safety and capability in general conditions. Mostly true: humans clearly led on general open-ended tasks, but in some narrow, limited deployments (for example a geofenced driverless robotaxi service) AI performance was arguably comparable to humans. Mixed: the evidence is roughly balanced on whether humans outperformed AI on open-ended tasks then. Mostly false: credible evidence then showed AI matching humans on several open-ended tasks, with humans leading on only a few. False: credible evidence then showed AI outperforming humans in general driving or similar open-ended tasks.

Set before the research began.

1. E1 Why hasn’t Waymo expanded its driverless service? Here’s my theory Ars Technica · 7 May 2021

   “Customers in a 50-square-mile corner of suburban Phoenix can now use their smartphones to hail a Chrysler Pacifica minivan with no one in the driver’s seat.”

2. E2 A secret weapon for self-driving car start-ups - humans The Straits Times (Reuters) · 31 August 2021

   “With no end in sight to the technical and regulatory obstacles to free-range, driverless robotaxis, some self-driving companies are accepting the need for human minders and scaling down their ambitions”

3. E3 Tesla privately admits Elon Musk has been exaggerating about ‘full self-driving’ The Verge · 7 May 2021

   “Tesla is unlikely to achieve Level 5 (L5) autonomy, in which its cars can drive themselves anywhere, under any conditions, without any human supervision, by the end of 2021, Tesla representatives told the DMV.”

4. E4 Calif. DMV Data Shows Autonomous Vehicles Improving - EE Times EE Times · 13 February 2021

   “Waymo was the top company at nearly 30,000 miles between disengagement on average, which is an improvement of over 126% from 2019.”

As of 2021, Waymo's driverless service was limited to a 50-square-mile area of suburban Phoenix (E1), and Reuters reported self-driving firms accepting the need for human minders amid ongoing technical obstacles (E2). Tesla told California regulators it was unlikely to reach full Level 5 autonomy by end of 2021 (E3). Waymo's disengagement figures (E4) show improvement but give no comparison showing better-than-human performance in general conditions, so the criteria for 'true' are met.