用卡尔曼滤波优化状态选择,实现长序列建模的上下文感知
KOSS: Kalman-Optimal Selective State Spaces for Long-Term Sequence Modeling
- 基于卡尔曼增益动态调节信息传播,实现闭环上下文感知选择
- 在干扰任务中准确率达79%以上,远超基线模型(<20%)
- 适用于雷达跟踪等实际场景,对噪声和不规则间隔有强鲁棒性
近期的选择性状态空间模型(SSMs),如Mamba和Mamba-2,凭借输入依赖的选择机制在序列建模中表现出色。然而,这些机制缺乏理论基础,无法基于潜在状态动态实现上下文感知选择。为此,本文提出KOSS,一种基于卡尔曼最优的选择性状态空间模型,将选择机制建模为潜在状态不确定性最小化。该模型基于估计理论,采用连续时间潜在状态更新,通过卡尔曼增益动态调节信息传播,实现闭环、上下文感知的选择性机制。为确保计算稳定与近线性可扩展性,KOSS引入全局频域微分进行导数估计,并采用分段扫描实现硬件高效处理。在含干扰项的选择复制任务中,KOSS准确率超过79%,而基线模型低于20%。在九个长期预测基准上,KOSS将均方误差降低2.92%至36.23%,在精度与稳定性上持续优于现有模型。针对实际应用,对二次监视雷达(SSR)跟踪的案例研究验证了其在非规则采样和噪声条件下的鲁棒性,展现了真实场景有效性。补充实验进一步验证了卡尔曼增益收敛性与频域微分的频率响应,为所提闭环设计提供理论支持。
原文摘要 · Abstract (English)
Recent selective state space models (SSMs), such as Mamba and Mamba-2, have demonstrated strong performance in sequence modeling owing to input-dependent selection mechanisms. However, these mechanisms lack theoretical grounding and cannot support context-aware selection from latent state dynamics. To address these limitations, we propose KOSS, a Kalman-optimal Selective State Space model that formulates selection as latent state uncertainty minimization. Derived from estimation theory, KOSS adopts a continuous-time latent update driven by a Kalman gain that dynamically modulates information propagation based on content and context, enabling a closed-loop, context-aware selectivity mechanism. To ensure stable computation and near-linear scalability, KOSS employs global spectral differentiation for frequency-domain derivative estimation, along with a segment-wise scan for hardware-efficient processing. On a selective copying task with distractors, KOSS achieves over 79\% accuracy while baselines drop below 20\%, demonstrating robust context-aware selection. Furthermore, across nine long-term forecasting benchmarks, KOSS reduces MSE by 2.92--36.23\% and consistently outperforms state-of-the-art models in both accuracy and stability. To assess real-world applicability, a case study on secondary surveillance radar (SSR) tracking confirms KOSS's robustness under irregular intervals and noisy conditions and demonstrates its effectiveness in real-world applications. Finally, supplementary experiments verify Kalman gain convergence and the frequency response of spectral differentiation, providing theoretical support for the proposed closed-loop design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。