As of July 2024, several tech companies are developing leading closed AI models.
Unresolvable The public record could not settle it.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
Said for an organisation they lead or work for; it counts on both records.
What would count as true or false
Judged as of 23 Jul 2024. True: at least three tech companies (e.g. OpenAI, Google/DeepMind, Anthropic, others) were then developing frontier AI models whose weights were not released, models that benchmarks or leaderboards (e.g. LMSYS Chatbot Arena, MMLU) ranked among the leading ones. Mostly true: this holds, but one of the companies counted is arguably not 'leading', or only about two to three clearly qualify. Mostly false: only one company had a leading closed model. False: no tech companies were developing leading closed models at that time.
Set before the research began.
Evidence (2)
-
E1 The Multimodal Arena is Here! LMSYS (Chatbot Arena) · 27 June 2024
“GPT-4o and Claude 3.5 achieve notably higher performance compared to Gemini 1.5 Pro and GPT-4 turbo. This gap is much more apparent in the vision arena compared to the language arena.”
-
E2 The Multimodal Arena is Here! LMSYS (Chatbot Arena) · 27 June 2024
“chat with your favorite vision-language models from OpenAI, Anthropic, Google, and most other major LLM providers”
Reasoning
The evidence shows that models from OpenAI, Anthropic and Google were compared with each other in the LMSYS arena in June 2024 [E1][E2]. It does not show where these models stood on the overall leaderboard, and it does not say their weights were unreleased, so the criteria for a leading closed model are not settled.
Cite this claim
Mark Zuckerberg, “As of July 2024, several tech companies are developing leading closed AI models.” (said 23 July 2024). Verdict: unresolvable. Reliability API, claim c_1729b469395e, method 0.1.0+3f84d14d, dataset release v1.1.2. https://epistacklabs.com/claims/c_1729b469395e
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Mark Zuckerberg
-
Resolved true
“It is bigger than avocado.”
-
Mostly true
“We're seeing early progress with the launch of our AI video editing tools across Meta AI and our new Edits app, and there's a lot more to do here.”
Meta launched AI video editing tools across Meta AI and its new Edits app.
-
Mostly true
“This year, Llama 3 is competitive with the most advanced models and leading in some areas.”
In 2024, Llama 3 is competitive with the most advanced models and leading in some areas.
-
Mostly true
“One that people are going to find pretty wild is that it now generates high quality images so quickly that it actually generates it as you're typing and updates it in real time.”
-
Resolved true
“we didn't prioritize coding because people aren't going to ask Meta AI a lot of coding questions in WhatsApp.”
Verdicts are made automatically; how verdicts are made.