Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
80 74 86 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
Insufficient record: 4 predictions resolved, 5 needed for a score.
-
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
85 74 94 -
Guidance
A company's own figures for its results over a period, checked against what it then reported.
Insufficient record: 0 guidance claims resolved, 5 needed for a score.
All types combined: 81 (likely 76 to 86, high confidence), from 86 claims: 70 factual claims, 16 promises. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Mark Zuckerberg's claims
“But even before that, Llama is already leading on openness, modifiability, and cost efficiency.”
Mark Zuckerberg, Meta CEO, writing on Meta Newsroom 23 July 2024 Open Source AI is the Path Forward (Meta Newsroom)
Llama is already leading on openness, modifiability, and cost efficiency.
Mixed Partly true and partly false in roughly equal measure.
The second judge would not resolve this claim; the verdict shown is the first judge's.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
This has three parts. True: as of July 2024, benchmarks or analyses show Llama leading frontier-class models on openness (open weights and permissive access), on modifiability (fine-tuning ecosystem, derivatives) and on cost efficiency (cost per token or performance per dollar). Mostly true: it leads on two and is near the top on the third. Mixed: it leads on one or two, but competitors (for example Mistral or more open models such as OLMo) match or beat it on others. Mostly false: it leads on at most one part, and comparable open models clearly beat it on the rest (for example open-source advocates rate its licence as less open than rivals'). False: it leads on none of the three.
Evidence
- With 10x growth since 2023, Llama is the leading engine of AI innovation Meta AI (ai.meta.com) Own record
“With more than 60,000 derivative models on Hugging Face, there’s a vibrant community of developers fine-tuning Llama for their own use cases.”
- Llama 3.1 405B, Meta’s AI strategy, and the new open frontier model ecosystem Interconnects (Nathan Lambert) · 23 July 2024
“The license is decent compared to Llama 3 and other recent open models, allowing synthetic data creation, but comes with heavy branding terms (more on this below).”
- Llama 3.1 405B, Meta’s AI strategy, and the new open frontier model ecosystem Interconnects (Nathan Lambert) · 23 July 2024
“Llama 3.1’s launch does not fit any of the proposed definitions of open-source AI, failing on the common step, data.”
- Meta’s LLaMa license is still not Open Source Open Source Initiative · 18 February 2025
“Llama 3.x is still not Open Source by any stretch of the imagination.”
- Llama 3.1 405B, Meta’s AI strategy, and the new open frontier model ecosystem Interconnects (Nathan Lambert newsletter) · 23 July 2024
“The Llama 3.1 models, which come in three sizes, 8, 70, and the long-rumored 405 billion parameters, are best-in-class open-weight models for their size.”
- Llama 3.1 405B, Meta’s AI strategy, and the new open frontier model ecosystem Interconnects (Nathan Lambert newsletter) · 23 July 2024
“The Llama 3.1 license is closer to freeware than open-source.”
- Llama 3.1 405B, Meta’s AI strategy, and the new open frontier model ecosystem Interconnects (Nathan Lambert newsletter) · 23 July 2024
“The 405B model pricing on APIs is very similar to GPT-4o, ranging from $3-9 per million input tokens and $3-15 per million output tokens where GPT-4o is $5 per million input, $15 per million output”
- Llama 3.1 405B, Meta’s AI strategy, and the new open frontier model ecosystem Interconnects (Nathan Lambert newsletter) · 23 July 2024
“With the strength of the evaluation scores of these models, training on synthetic data from Llama 3.1 Instruct 405B will be much cheaper than using an API at most labs, and fine-tuning from the 8 or 70B models will be most academic’s core baseline and starting point.”
- Llama 3.1 405B vs Mistral Large 2 (2407) (2024) | LLMReference LLMReference
“| Commercial use | Restricted | Not permitted |”
- Llama 3.1 405B vs Mistral Large 2 (2407) (2024) | LLMReference LLMReference
“| Openness | Open weights | Open weights |”
- Llama 3.1 405B vs Mistral Large 2 (2407) (2024) | LLMReference LLMReference
“Llama 3.1 405B is listed under Llama 3 Community. Mistral Large 2 (2407) is listed under Mistral License.”
- Llama 3.1 405B vs Mistral Large 2 (2407) (2024) | LLMReference LLMReference
“Llama 3.1 405B has no token price listed and Mistral Large 2 (2407) has $0.50/1M input tokens.”
Reasoning
On openness, independent commentary from the claim date says the Llama 3.1 licence is "closer to freeware than open-source" and fails the proposed open-source AI definitions on data (E3, E6). The OSI later said Llama 3.x is not open source (E4). Even so, Llama's licence is rated decent next to other recent open models (E2), and a comparison table appears to list Llama's commercial use as restricted against not permitted for Mistral Large 2, though its columns are unlabelled (E9). On modifiability, E8 expects the 8B and 70B models to become the standard starting point for fine-tuning, and Meta reports more than 60,000 derivative models (E1), though that count comes from Meta itself. On cost efficiency, E7 finds 405B API pricing "very similar to GPT-4o" rather than clearly cheaper, so the evidence does not show Llama leading on all three parts.
Cite this claim
Mark Zuckerberg, “Llama is already leading on openness, modifiability, and cost efficiency.” (said 23 July 2024). Verdict: mixed. Reliability API, claim c_4941dea23f5d, method 0.1.0+3f84d14d. https://epistacklabs.com/claims/c_4941dea23f5d
Claims without a verdict
- Unresolvable 144
- Not yet due 17
- Own records only 41
- Not researched 223
- Not checkable 1,060
Is a verdict here wrong, or do you speak for Mark Zuckerberg? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.