The Mamba architecture can think for more than 200 serial steps per thought.
Unresolvable The public record could not settle it.
Factual: Statements about the present or the past, checked against the record: did they hold up?
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judge as of 2024-03-15. True: technical descriptions of Mamba (Gu & Dao 2023) show that its recurrent state-space computation allows serial computation depth that grows with sequence length, beyond the fixed depth of a transformer forward pass (on the order of the number of layers, around 200 or fewer), so it can carry out more than 200 serial steps per output. Mostly true: this holds broadly, with technical caveats (e.g. the selective-scan recurrence is linear within a layer, so how much 'serial computation' it adds is debatable). Mixed: experts disagree substantially on whether Mamba's recurrence amounts to added serial reasoning depth. Mostly false: the analysis shows Mamba's effective serial depth is bounded much like a transformer's, with only a narrow sense in which the claim holds. False: Mamba clearly cannot perform more serial steps than a fixed-depth transformer.
Set before the research began.
Evidence (3)
-
E1 Mamba: Linear-Time Sequence Modeling with Selective State Spaces arXiv (Gu & Dao)
“We overcome this with a hardware-aware algorithm that computes the model recurrently with a scan instead of convolution, but does not materialize the expanded state in order to avoid IO access between different levels of the GPU memory hierarchy.”
-
E2 The Illusion of State in State-Space Models National Science Foundation Public Access Repository (PAR) · 4 July 2024
“Our analysis reveals that the expressive power of SSMs is limited very similarly to transformers: SSMs cannot express computation outside the complexity class 𝖳𝖢0.”
-
E3 On Recent Results in LLM Latent Reasoning — LessWrong LessWrong · 31 March 2025 Own record
“A few months ago, I was surprised to find that most state-space models choose the ‘parallelizable training’ side of the trade-off, and that the effective hidden serial reasoning depth of architectures like Mamba and RWKV therefore isn’t substantially higher than that of transformers.”
Reasoning
E1's quoted passage describes Mamba's recurrent scan but does not address serial reasoning depth. E2 is an independent result about the expressive power of SSMs in general, not about Mamba's serial depth, and it postdates the judgment date. The only source that speaks directly to serial depth is E3, which also postdates the judgment date and is not independent of the subject, so the independent evidence does not settle the claim under the criteria.
Cite this claim
Eliezer Yudkowsky, “The Mamba architecture can think for more than 200 serial steps per thought.” (said 15 March 2024). Verdict: unresolvable. Reliability API, claim c_06449044658c, method 0.1.0+c876030c, dataset release v1.1.2. https://epistacklabs.com/claims/c_06449044658c
This claim is in dataset release v1.1.2, with the same verdict.
Reply to this claim
More factual claims by Eliezer Yudkowsky
-
Resolved true
“We learned that that's false, but we did not learn that you can have a thing that can redesign a tree and build an AI much smarter than itself, that builds an AI much smarter than itself, and that…”
-
Mostly true
“I was a passionate noticer of the human brain not being homogenous, and was therefore unpleasantly surprised by transformers scaling as well as they did.”
-
Mostly true
“First of all, you realize in principle that the task of minimizing losses on predicting human text does not stop when you’re as smart as a human, right?”
-
Mostly true
“Transformers are the main thing like that.”
-
Mostly true
“Because the basic paradigm of machine learning that we are presently operating under is that you can have the loss function, but only for things you can evaluate.”
Verdicts are made automatically; how verdicts are made.