提出可调控记忆行为的高效视觉状态空间模型,提升长程建模能力。
TCP-SSM: Efficient Vision State Space Models with Token-Conditioned Poles

- 通过可调节极点显式控制递归动态,实现稳定记忆机制。
- 在保持线性复杂度下,计算量减少最多44%,精度不降反升。
- 适合追求高效长程视觉建模的开发者,尤其适用于紧凑模型设计。
状态空间模型(SSMs)作为长序列视觉任务中注意力机制的有力替代方案,具备输入相关的递归特性与线性复杂度。然而,现有高效SSM变体多通过修改扫描路径、分辨率或遍历模式来降低计算开销,而将递归动态隐式化,导致状态依赖的记忆行为难以控制,尤其在紧凑骨干网络中,长扫描路径可能超出有效记忆范围。本文提出一种结构化选择性状态空间模型——令牌条件极点SSM(TCP-SSM),通过引入稳定极点使递归动态显式且可解释。每个扫描算子由实极点(模拟单调或符号交替衰减)和共轭复极点(捕捉阻尼振荡响应)构成。利用有界半径与角度调制,将共享基础极点转换为令牌相关极点,使每一步扫描能根据当前视觉令牌自适应调整记忆行为,同时保证极点稳定性。为提升实际可扩展性,引入分组极点共享与轻量化低秩输入路径,构建出高效的扫描算子,维持线性时间扫描复杂度。在图像分类、语义分割和目标检测任务中,相较于Vision Mamba类模型,TCP-SSM将计算复杂度降低最多达44%,并保持或超越基线精度。
原文摘要 · Abstract (English)
State Space Models (SSMs) have emerged as a compelling alternative to attention models for long-range vision tasks, offering input-dependent recurrence with linear complexity. However, most efficient SSM variants reduce computation cost by modifying scan routes, resolutions, or traversal patterns, while largely leaving the recurrent dynamics implicit. Consequently, the model's state-dependent memory behavior is difficult to control, particularly in compact backbones where long scan paths can exceed the effective memory horizon. We propose Token-Conditioned Poles SSM (TCP-SSM), a structured selective SSM framework that improves efficiency while making recurrence dynamics explicit and interpretable through stable poles. TCP-SSM builds each scan operator with 1) real poles that model monotone or sign-alternating decay, and 2) complex-conjugate poles that capture damped oscillatory responses. Using bounded radius and angle modulation, TCP-SSM converts shared base poles into token-dependent poles, allowing each scan step to adapt its memory behavior to the current visual token while preserving pole stability. For practical scalability, we integrate grouped pole sharing with a lightweight low-rank input pathway, yielding an efficient scan operator that preserves linear-time scan complexity. Across image classification, semantic segmentation, and object detection, TCP-SSM reduces SSM computation complexity up to 44% in Vision Mamba-style models while maintaining or surpassing baseline accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。