Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
79 72 85 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
80 72 88 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
Insufficient record: 1 promise resolved, 5 needed for a score.
All types combined: 79 (likely 74 to 84, high confidence), from 102 claims: 64 factual claims, 38 predictions. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Gary Marcus's claims
42 claims
“Immediately after the February announcement of Sora, I noted Sora’s troubles with physics and predicted that they wouldn’t go away. They haven’t.”
Gary Marcus, Substack author February 2024 25 AI Predictions for 2025, from Marcus on AI (Marcus on AI)
After Sora's February 2024 announcement, Marcus predicted its physics troubles would persist, and they have.
Resolved true The record shows it happened or was so.
What would count as true or false
Two parts, judged as of the date Marcus published this statement (after February 2024, likely later in 2024). (a) Immediately after OpenAI announced Sora (15 February 2024), Marcus noted its physics problems and predicted they would persist. (b) Those physics problems had not gone away by the time of the statement. True: dated Marcus posts from within days of the announcement contain both the observation and the prediction, and evidence up to the statement date (released Sora outputs, reviews, analyses) shows Sora still producing physically implausible results. Mostly true: both parts hold, but a minor detail differs, for example the prediction came some days or weeks later, or the problems had lessened noticeably while remaining evident. Mixed: one part holds and the other does not, for example he noted the problems but made no prediction. Mostly false: he made the prediction, but the physics problems had largely been resolved by the time of the statement. False: he made no such observation or prediction around the announcement, and the problems were resolved.
Evidence
- Sora’s Surreal Physics Gary Marcus (Substack) · 16 February 2024 Own record
“And importantly, I predict that many will be hard to remedy.”
- Sora is here OpenAI · 9 December 2024
“It often generates unrealistic physics and struggles with complex actions over long durations.”
- Sora still appears to have trouble with physics Gary Marcus (Substack) · 10 December 2024 Own record
“Looks like these problems linger in a new release of Sora that came out today”
- No, Sora has not “learned physics” Gary Marcus (Substack) · 2 March 2024 Own record
“It is precisely because Sora is not a physics engine that a large fraction of Sora's videos are inconsistent with physics or biology, in very basic ways.”
Reasoning
Marcus's own post dated 16 February 2024, the day after the Sora announcement, is a dated record of what he wrote then. In it he noted physical glitches and predicted that many would be "hard to remedy" (E1), and he repeated the point on 2 March 2024 (E4). OpenAI is independent of Marcus, and its December 2024 release announcement says Sora "often generates unrealistic physics" (E2). That shows the problems were still there in 2024, which matches Marcus's December 2024 observation (E3).
Cite this claim
Gary Marcus, “After Sora's February 2024 announcement, Marcus predicted its physics troubles would persist, and they have.” (said February 2024). Verdict: resolved true. Reliability API, claim c_c25cce7c54bc, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_c25cce7c54bc
Claims without a verdict
Is a verdict here wrong, or do you speak for Gary Marcus? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.