As DL is used for high-level cognitive tasks, memory modules will become commonplace and very large, requiring hardware support.

Judge at evaluation time; there is no stated horizon. 'Memory modules' means explicit associative or key-value memory components in deep-learning systems: memory-network-style key-value memories, retrieval memories, and attention key-value stores such as transformer KV caches. The claim has three parts: (a) such modules become commonplace in DL systems used for high-level cognitive tasks such as language, reasoning and dialogue; (b) they become very large (for example gigabytes, or millions of entries or tokens and more); (c) they need, and receive, hardware support, such as hardware, memory-system or accelerator features designed for these memory workloads. True: all three hold. Mostly true: all three hold, but one only in a weaker form (for example the hardware support is mainly software or system-level optimisation on general accelerators). Mixed: two of the three hold. Mostly false: only one holds. False: explicit memory modules remain uncommon in such systems.

Set before the research began.

1. E1 From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving arXiv (academic survey by university authors)

   “The key-value (KV) cache has become a first-order memory object in LLM serving rather than a temporary per-request tensor.”

2. E2 Accelerate Large-Scale LLM Inference and KV Cache Offload with CPU-GPU Memory Sharing | NVIDIA Technical Blog NVIDIA · 5 September 2025

   “A KV-cache representing a 128k token context window for a single user (batch size 1) consumes about 40 GB of memory with Llama 3 70B, and this scales linearly with the number of users.”

3. E3 PAM: Processing Across Memory Hierarchy for Efficient KV-centric LLM Serving System arXiv (academic research paper)

   “For instance, processing 256 OPT-175B requests with 2048-token contexts requires approximately 2304 GB of KV cache, which far exceeds the capacity of platforms like DGX-H100 (NVIDIA, 2022).”

4. E4 Kelle: Co-design KV Caching and eDRAM for Efficient LLM Serving in Edge Computing arXiv (academic research paper)

   “To mitigate the substantial overhead caused by KV cache, we propose using embedded DRAM (eDRAM) as the primary storage for LLM serving in edge device, which offers higher storage density compared to SRAM.”

E1, a survey, says the KV cache has become a first-order memory object in LLM serving, which supports part (a): such modules are commonplace. E2 puts the KV cache for one 128k-token context on Llama 3 70B at about 40 GB, and E3 gives about 2,304 GB for 256 OPT-175B requests, more than a DGX-H100 can hold, which supports part (b): they are very large. For part (c), E3 and E4 propose hardware designs aimed at KV-cache workloads (processing across the memory hierarchy, and eDRAM co-design). E2's title, however, describes KV cache offload using CPU-GPU memory sharing on general accelerators. So the hardware support shown is the weaker form: research proposals and system-level methods rather than widely deployed dedicated hardware, which makes the claim mostly true.