用Mamba-2替换Transformer，让小模型更稳地解决抽象推理题。
Tiny Recursive Reasoning with Mamba-2 Attention Hybrid
- 用Mamba-2混合算子替代TRM中的Transformer块，保持参数量相近。
- 在ARC-AGI-1上pass@2提升2.0%（45.88%），pass@100提升4.75%。
- 证明状态空间模型适合递归推理，为高效架构设计提供新方向。
近期研究显示，小型网络（700万参数）通过隐状态空间的递归迭代优化，可在抽象推理任务中取得优异表现。这引出一个关键问题：能否将具备内在迭代特性的Mamba-2状态空间模型引入递归框架？本文将TRM中的Transformer模块替换为Mamba-2混合算子，在参数量几乎一致（683万 vs 686万）的前提下进行测试。在ARC-AGI-1数据集上，该混合模型的pass@2指标提升2.0%（从43.88%增至45.88%），且在更高K值下表现更优（pass@100提升4.75%），同时维持pass@1性能不变。结果表明，模型生成正确解的能力显著增强，而最优选择能力保持稳定。验证了基于SSM的算子可有效保留递归推理能力，为未来递归结构设计提供了新范式。
原文摘要 · Abstract (English)
Recent work on recursive reasoning models like TRM demonstrates that tiny networks (7M parameters) can achieve strong performance on abstract reasoning tasks through latent recursion -- iterative refinement in hidden representation space without emitting intermediate tokens. This raises a natural question about operator choice: Mamba-2's state space recurrence is itself a form of iterative refinement, making it a natural candidate for recursive reasoning -- but does introducing Mamba-2 into the recursive scaffold preserve reasoning capability? We investigate this by replacing the Transformer blocks in TRM with Mamba-2 hybrid operators while maintaining parameter parity (6.83M vs 6.86M parameters). On ARC-AGI-1, we find that the hybrid improves pass@2 (the official metric) by +2.0\% (45.88\% vs 43.88\%) and consistently outperforms at higher K values (+4.75\% at pass@100), whilst maintaining pass@1 parity. This suggests improved candidate coverage -- the model generates correct solutions more reliably -- with similar top-1 selection. Our results validate that Mamba-2 hybrid operators preserve reasoning capability within the recursive scaffold, establishing SSM-based operators as viable candidates in the recursive operator design space and taking a first step towards understanding the best mixing strategies for recursive reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。