The attached image shows the computing infrastructure used for Llama-3 training.

Judge as of 2024-03-12. True: the image LeCun posted depicts Meta's AI computing infrastructure that Meta stated would be or was used to train Llama 3 (e.g. the 24k-GPU H100 clusters Meta announced on 2024-03-12 for Llama 3). Mostly true: it shows Meta's GenAI training infrastructure that includes the Llama 3 clusters, but the specific hardware shown is generic or a rendering rather than the actual cluster. Mixed: it shows Meta infrastructure whose link to Llama 3 is unclear. Mostly false: it shows Meta data-center hardware not tied to Llama 3 training (e.g. older RSC hardware). False: it shows unrelated infrastructure or something other than computing infrastructure.

Set before the research began.

1. E1 Building Meta’s GenAI Infrastructure Meta Engineering (engineering.fb.com) · 12 March 2024

   “We use this cluster design for Llama 3 training.”

2. E2 Building Meta’s GenAI Infrastructure Meta Engineering (engineering.fb.com) · 12 March 2024

   “Through careful co-design of the network, software, and model architectures, we have successfully used both RoCE and InfiniBand clusters for large, GenAI workloads (including our ongoing training of Llama 3 on our RoCE cluster) without any network bottlenecks.”

3. E3 Meta unveils two new 24K GPU clusters it's using to develop more advanced generative AI - SiliconANGLE SiliconANGLE · 12 March 2024

   “Meta Platforms Inc. today revealed a pair of enormously powerful graphics processing unit clusters that it says will be used to support the training of next-generation generative artificial intelligence models, including the upcoming Llama 3.”

4. E4 Meta intros two GPU training clusters for Llama 3 TechTarget · 12 March 2024

   “Meta on Tuesday introduced two 24K GPU clusters that it is using to train its forthcoming Llama 3 LLM.”

5. E5 Meta unveils two new GPU clusters used to train its Llama 3 AI model — and it plans to acquire an extra 350,000 Nvidia H100 GPUs by the end of 2024 to meet development goals ITPro · 13 March 2024

   “These GPUs will support both Meta’s current Llama 2 model and its upcoming Llama 3 model, as well as the company’s wider research and development projects across generative AI and other areas.”

6. E6 Meta Unveils 24k GPU AI Infrastructure Design InfoQ · 2 April 2024

   “Meta recently announced the design of two new AI computing clusters, each containing 24,576 GPUs. The clusters are based on Meta's Grand Teton hardware platform, and one cluster is currently used by Meta for training their next-generation Llama 3 model.”

7. E7 Introducing Meta Llama 3: The most capable openly available LLM to date Meta AI Own record

   “We performed training runs on two custom-built 24K GPU clusters.”

8. E8 Meta unveils two new GPU clusters used to train its Llama 3 AI model — and it plans to acquire an extra 350,000 Nvidia H100 GPUs by the end of 2024 to meet development goals ITPro · 13 March 2024

   “With high-performance network fabrics working alongside 24,576 Nvidia Tensor Core H100 GPUs, these new clusters are able to support “larger and more complex” models than Meta’s previous RSC clusters.”

Independent sources confirm that on 2024-03-12 Meta announced two 24K-GPU clusters it was using to train Llama 3 (E1, E2, E3, E4, E6). However, none of the evidence describes the image LeCun posted. Nothing shows whether it depicts those clusters, a generic rendering, older RSC hardware or something else. The criteria depend on what the image shows, so the evidence does not settle the claim.