Recent news: Gated Delta Networks and Mamba-3

We are wrapping up our overview of SSM-based models. This time, we will cover two recent extensions of this line of work: Gated Delta Networks (ICLR 2025) and Mamba-3 (submitted to ICLR 2026). Both models extend the ideas of linear recurrent architectures and demonstrate how SSMs can compete with transformers on increasingly complex tasks.

Gated DeltaNet is built on top of Mamba-2 and combines two approaches to memory control — “gating” and the “delta rule.” This allows the model to decide what to fully forget and what to update selectively, while maintaining efficiency and adaptability to long contexts.

Mamba-3 takes the next step by rethinking the very structure of SSMs from the perspective of efficient inference. It introduces more expressive recurrent dynamics, complex-valued state updates, and a MIMO (multi-input, multi-output) formulation, making it faster and more powerful without increasing the computational cost of generation.