36 claims
“But even before that, Llama is already leading on openness, modifiability, and cost efficiency.”
Mark Zuckerberg, Meta CEO, writing on Meta Newsroom 23 July 2024 Open Source AI is the Path Forward (Meta Newsroom)
Llama is already leading on openness, modifiability, and cost efficiency.
Mixed Partly true and partly false in roughly equal measure.
The second judge would not resolve this claim; the verdict shown is the first judge's.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
This has three parts. True: as of July 2024, benchmarks or analyses show Llama leading frontier-class models on openness (open weights and permissive access), on modifiability (fine-tuning ecosystem, derivatives) and on cost efficiency (cost per token or performance per dollar). Mostly true: it leads on two and is near the top on the third. Mixed: it leads on one or two, but competitors (for example Mistral or more open models such as OLMo) match or beat it on others. Mostly false: it leads on at most one part, and comparable open models clearly beat it on the rest (for example open-source advocates rate its licence as less open than rivals'). False: it leads on none of the three.
Evidence
- With 10x growth since 2023, Llama is the leading engine of AI innovation Meta AI (ai.meta.com) Own record
“With more than 60,000 derivative models on Hugging Face, there’s a vibrant community of developers fine-tuning Llama for their own use cases.”
- Llama 3.1 405B, Meta’s AI strategy, and the new open frontier model ecosystem Interconnects (Nathan Lambert) · 23 July 2024
“The license is decent compared to Llama 3 and other recent open models, allowing synthetic data creation, but comes with heavy branding terms (more on this below).”
- Llama 3.1 405B, Meta’s AI strategy, and the new open frontier model ecosystem Interconnects (Nathan Lambert) · 23 July 2024
“Llama 3.1’s launch does not fit any of the proposed definitions of open-source AI, failing on the common step, data.”
- Meta’s LLaMa license is still not Open Source Open Source Initiative · 18 February 2025
“Llama 3.x is still not Open Source by any stretch of the imagination.”
- Llama 3.1 405B, Meta’s AI strategy, and the new open frontier model ecosystem Interconnects (Nathan Lambert newsletter) · 23 July 2024
“The Llama 3.1 models, which come in three sizes, 8, 70, and the long-rumored 405 billion parameters, are best-in-class open-weight models for their size.”
- Llama 3.1 405B, Meta’s AI strategy, and the new open frontier model ecosystem Interconnects (Nathan Lambert newsletter) · 23 July 2024
“The Llama 3.1 license is closer to freeware than open-source.”
- Llama 3.1 405B, Meta’s AI strategy, and the new open frontier model ecosystem Interconnects (Nathan Lambert newsletter) · 23 July 2024
“The 405B model pricing on APIs is very similar to GPT-4o, ranging from $3-9 per million input tokens and $3-15 per million output tokens where GPT-4o is $5 per million input, $15 per million output”
- Llama 3.1 405B, Meta’s AI strategy, and the new open frontier model ecosystem Interconnects (Nathan Lambert newsletter) · 23 July 2024
“With the strength of the evaluation scores of these models, training on synthetic data from Llama 3.1 Instruct 405B will be much cheaper than using an API at most labs, and fine-tuning from the 8 or 70B models will be most academic’s core baseline and starting point.”
- Llama 3.1 405B vs Mistral Large 2 (2407) (2024) | LLMReference LLMReference
“| Commercial use | Restricted | Not permitted |”
- Llama 3.1 405B vs Mistral Large 2 (2407) (2024) | LLMReference LLMReference
“| Openness | Open weights | Open weights |”
- Llama 3.1 405B vs Mistral Large 2 (2407) (2024) | LLMReference LLMReference
“Llama 3.1 405B is listed under Llama 3 Community. Mistral Large 2 (2407) is listed under Mistral License.”
- Llama 3.1 405B vs Mistral Large 2 (2407) (2024) | LLMReference LLMReference
“Llama 3.1 405B has no token price listed and Mistral Large 2 (2407) has $0.50/1M input tokens.”
Reasoning
On openness, independent commentary from the claim date says the Llama 3.1 licence is "closer to freeware than open-source" and fails the proposed open-source AI definitions on data (E3, E6). The OSI later said Llama 3.x is not open source (E4). Even so, Llama's licence is rated decent next to other recent open models (E2), and a comparison table appears to list Llama's commercial use as restricted against not permitted for Mistral Large 2, though its columns are unlabelled (E9). On modifiability, E8 expects the 8B and 70B models to become the standard starting point for fine-tuning, and Meta reports more than 60,000 derivative models (E1), though that count comes from Meta itself. On cost efficiency, E7 finds 405B API pricing "very similar to GPT-4o" rather than clearly cheaper, so the evidence does not show Llama leading on all three parts.
Cite this claim
Mark Zuckerberg, “Llama is already leading on openness, modifiability, and cost efficiency.” (said 23 July 2024). Verdict: mixed. Reliability API, claim c_4941dea23f5d, method 0.1.0+3f84d14d. https://epistacklabs.com/claims/c_4941dea23f5d
Verdicts are made automatically; how verdicts and scores are made. The same data is on Hugging Face under CC BY 4.0.