Current-sized LLMs are expensive and not reliable enough for many jobs.

Judge as of 2024-03-31. The claim has two parts: (a) current-sized frontier LLMs as of March 2024 were expensive to train and/or run, and (b) they were not reliable enough for many jobs (for example, error or hallucination rates too high for deployment in a substantial number of occupational tasks). True: contemporaneous evidence (cost estimates for training or inference, industry reports, studies of hallucination and error rates, documented deployment failures or companies limiting LLM use because of accuracy) supports both parts. Mostly true: both parts are broadly supported, but one is somewhat overstated, for example costs were falling fast or reliability was adequate for a large share of tasks while still falling short for many. Mixed: one part is well supported and the other is not. Mostly false: the evidence shows LLMs were already reliable enough for most of the jobs at issue and cheap enough for wide use, with only narrow support for the claim. False: the evidence shows that in March 2024 such models were both inexpensive and reliable across most jobs.

Set before the research began.

1. E1 Why the cost of training AI could soon become too much to bear | Fortune Fortune · 4 April 2024

   “Although companies like OpenAI and Google don’t disclose the precise costs of training AI models like GPT-4 and Gemini, it’s clearly a fiendishly expensive business—and the bigger and more capable these so-called frontier models get, the more it costs to train them.”

2. E2 IT Spending Pulse: As GenAI Investment Grows, Other IT Projects Get Squeezed Boston Consulting Group · 1 July 2024

   “The leading challenge is the immaturity of GenAI technology, which was cited as a barrier by 43% of high maturity, 36% of mid maturity, 38% of low maturity, and 50% of companies with little or no maturity.”

3. E3 Colorado Lawyer Filed a Motion Citing Hallucinated ChatGPT Cases (2023) – AI Incident 615 MIT AI Risk Repository (AI Incident Database entry) · 1 January 2026

   “The model hallucinated several non-existent cases, which the attorney submitted to the court without verification.”

4. E4 LLM inference prices have fallen rapidly but unequally across tasks Epoch AI · 12 March 2025

   “The inference price of LLMs has fallen dramatically in recent years.”

E1, published a few days after the judging date, describes training frontier models such as GPT-4 and Gemini as "fiendishly expensive," which supports the cost part. E4, a 2025 analysis, reports that LLM inference prices have fallen dramatically in recent years, which may temper the cost part for running models, though it post-dates the judging date and does not state what running the models cost in March 2024. On reliability, E2 reports that companies cited the immaturity of GenAI technology as their leading barrier to adoption. E3 documents a workplace case in which hallucinated legal citations were filed in court. Together these broadly support the reliability part, but they are indirect and anecdotal rather than measured error rates across jobs.