Meta is always training another model beyond its current released one.

Judged as of 2023-09-28. True: evidence shows that Meta, as standard practice, was training a successor or further model beyond its latest released one around that time. Examples would be Llama 3 or other models in training after Llama 2 was released in July 2023, consistent with continuous training. Mostly true: Meta was training a next model then, but 'always' slightly overstates it (for example, there were brief gaps). Mixed: training was going on only intermittently. Mostly false: there is little evidence of a further model being trained at that time. False: evidence shows Meta was not training any further model then.

Set before the research began.

1. E1 Meta unveils two new 24K GPU clusters it's using to develop more advanced generative AI - SiliconANGLE SiliconANGLE · 12 March 2024

   “Meta said it will use the new clusters to fine-tune its existing AI systems and train newer, more powerful ones, including Llama 3, the planned successor to Llama 2.”

2. E2 Meta unveils two new 24K GPU clusters it's using to develop more advanced generative AI - SiliconANGLE SiliconANGLE · 12 March 2024

   “The blog post marks the first time that Meta has confirmed it is working on Llama 3, though it was widely suspected of doing so.”

3. E3 META Q3 2023 Prepared Remarks Meta Platforms, Inc. · 25 October 2023 Own record

   “on top of that we're also building foundation models like Llama 2, which we believe is now the leading open source model, with more than 30 million Llama downloads last month.”

4. E4 Meta Platforms, Inc. (META) Third Quarter 2023 Results – Prepared Remarks Meta Platforms, Inc. · 25 October 2023 Own record

   “we’re also building foundation models like Llama 2, which we believe is now the leading open source model, with more than 30 million Llama downloads last month.”

5. E5 Meta Platforms, Inc. (META) Third Quarter 2023 Results – Prepared Remarks Meta Platforms, Inc. · 25 October 2023 Own record

   “Our investments in AI capacity in recent years have enabled us to support leading Generative AI research and we’re meaningfully accelerating the pace of bringing that research into our products.”

6. E6 Meta Platforms, Inc. (META) Third Quarter 2023 Results – Prepared Remarks Meta Platforms, Inc. · 25 October 2023 Own record

   “we expect AI to be the largest area of increased investment as we further invest in Generative AI across our core products, internal tooling, and research efforts.”

7. E7 The Week in Generative AI: September 15, 2023 edition Quad Insights · 15 September 2023

   “Meta “is working on a new artificial-intelligence system intended to be as powerful as the most advanced model offered by OpenAI.””

8. E8 The Week in Generative AI: September 15, 2023 edition Quad Insights · 15 September 2023

   “Unlike its partnership with Microsoft for Llama 2, Meta plans to use its own infrastructure for training the new model in early 2024, using advanced Nvidia H100 chips.”

9. E9 Privacy Matters: Meta’s Generative AI Features Meta (about.fb.com) · 27 September 2023 Own record

   “we included additional privacy safeguards and integrity mitigations as we built new models and used information from our products and services.”

10. E10 Building Meta’s GenAI Infrastructure Meta Engineering (engineering.fb.com) · 12 March 2024 Own record

    “These clusters support our current and next generation AI models, including Llama 3, the successor to Llama 2, our publicly released LLM”

The evidence shows Meta was working on a new AI system in September 2023 (E7) and confirmed work on Llama 3 in March 2024 (E1, E2, E10). However, it also reports that Meta planned to train the new model in early 2024 (E8), and no item shows a further model actually being trained around 2023-09-28.