用几何控制理论让线性时不变模型实现信息选择，性能超越Mamba。
Geometric SSM: LTI State Space Models for Selective Tasks
- 通过输入激发动态系统的不同不变子空间实现选择性
- 在扩展归纳头任务上接近完美表现，而Mamba失败
- 保留快速FFT训练优势，无需时变系统矩阵
近期关于选择性状态空间模型的研究认为，选择性需打破线性时不变（LTI）特性，依赖时变动态。本文挑战此观点，证明在几何控制理论指导下，LTI系统同样可实现选择性。提出几何状态空间模型（Geometric SSM），不同输入模式会激发系统动态的不同不变子空间。与Mamba无记忆选择机制不同，该方法采用动态残差生成器，保持时间记忆，可识别多标记模式，且无需时变系统矩阵。在新提出的扩展归纳头任务中，几何SSM表现接近完美，而Mamba失败；同时维持基于FFT的高效训练。结果表明，几何控制理论可指导设计兼具理论严谨性与实际效率的新一代选择性序列模型。
原文摘要 · Abstract (English)
A key claim in recent work on Selective State Space Models is that selectivity, the ability to focus on relevant information while filtering irrelevant inputs, requires breaking the Linear Time-Invariant (LTI) property through time-varying dynamics. We challenge this claim by demonstrating that LTI systems can achieve selectivity when designed using principles from geometric control. We introduce the Geometric SSM, in which different input patterns excite distinct invariant subspaces of the dynamics. Unlike Mamba's memoryless selection mechanism, our approach employs a dynamic residual generator that maintains temporal memory, enabling recognition of multi-token patterns without time-varying system matrices. The Geometric SSM achieves near-perfect performance on a novel extended induction head task where Mamba fails, while preserving efficient FFT-based training. Our results demonstrate that geometric control theory can inform the design of novel selective sequence models that combine theoretical rigor with practical efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。