Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
79 72 85 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
80 72 88 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 1 promise resolved, 5 needed for a score.
All types combined: 79 (likely 74 to 84, high confidence), from 102 claims: 64 factual claims, 38 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Gary Marcus's claims
52 claims
“Sora will continue to have trouble with physics.”
Gary Marcus, Substack author 1 January 2025 25 AI Predictions for 2025, from Marcus on AI (Marcus on AI)
OpenAI's Sora will continue to have trouble with physics in 2025.
Mostly true True in substance, wrong in a detail.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
This prediction is judged over calendar year 2025. It covers any OpenAI video model released under the Sora name (including Sora 2). TRUE: through 2025, credible evidence shows Sora outputs still commonly break physical plausibility. Examples are objects merging or appearing from nowhere, wrong collisions, gravity or fluid errors, and body or limb artifacts. The evidence can be independent reviews, physics-focused benchmarks or evaluations, widely documented examples, or OpenAI's own acknowledgment of physics limits. The problems must persist into the latter part of 2025, including for the newest Sora version available then. MOSTLY TRUE: physics errors are clearly documented in 2025 but are described as markedly less frequent or confined to harder scenes in the latest version. MOSTLY FALSE: physics problems were documented early in 2025, but by late 2025 credible evaluations show the then-current Sora handles physics reliably and errors are rare. FALSE: credible evaluations across 2025 show Sora consistently produces physically accurate video and physics errors are not a notable problem.
Evidence
- Sora 2 is here OpenAI · 30 September 2025 Own record
“Interestingly, “mistakes” the model makes frequently appear to be mistakes of the internal agent that Sora 2 is implicitly modeling; though still imperfect, it is better about obeying the laws of physics compared to prior systems.”
- Sora and The Big Bright Screen Slop Machine Zvi Mowshowitz (Don't Worry About the Vase, Substack) · 3 October 2025
“It does still fail in ways that can look embarrassing. For example, here we have it successfully having a ball respond to gravity when the ball is white, then have it all go wrong when the ball is red.”
- Sora 2 is mind-blowing... LilysAI (AI-generated note summarising a YouTube video) · 11 November 2025
“Milk was poured into dark coffee, but the expected plume did not appear.”
- PhyWorldBench: A Comprehensive Evaluation of Physical Realism in Text-to-Video Models arXiv (academic paper by external researchers)
“Well-performed models like Sora, Gen-3, and Pika often introduce exaggerated motion dynamics, such as objects moving with heightened fluidity or unnatural acceleration, making scenes appear choreographed rather than physically grounded.”
Reasoning
At the Sora 2 launch in September 2025, OpenAI said the model was "still imperfect" but "better about obeying the laws of physics compared to prior systems" (E1). An October 2025 review describes a Sora 2 ball that responds correctly to gravity when white but goes wrong when red (E2), and a November 2025 AI-generated summary of a video review reports that the expected plume did not appear when milk was poured into coffee (E3). PhyWorldBench (E4, undated) finds that Sora exaggerates motion dynamics in ways that are not physically grounded. So physics errors in the newest version are documented into late 2025. But the evidence is mostly individual examples rather than proof that errors were common, and OpenAI describes clear improvement, which fits the mostly-true criterion better than the true one.
Cite this claim
Gary Marcus, “OpenAI's Sora will continue to have trouble with physics in 2025.” (said 1 January 2025). Verdict: mostly true. Reliability API, claim c_f78e68dfbc91, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_f78e68dfbc91
Claims without a verdict
Is a verdict here wrong, or do you speak for Gary Marcus? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.