用新型扫描结构让模型自动生成运动特征，提升视频动作识别精度。
MamBOA: State-Space Architecture for Video Recognition
- 通过交错扫描重构状态空间，让运动成为隐状态的内在动态。
- 在Diving48数据集上达85.02%（图像预训练）和86.24%（视频预训练）准确率。
- 兼容CNN、Transformer、Mamba，仅增加约2.1 GFLOPs每对特征。
细粒度动作识别需要时间推理能力，通用架构通过不同成本-精度权衡实现：3D密集算子将计算与输入体积耦合，而基于差分的方法通过刚性手工减法近似运动，且不依赖上下文特征——各自体现设计取舍，限制了表达力或灵活性。本文提出MamBOA，一种无需依赖主干网络的时间框架，基于新颖的交错扫描结构，将选择性状态空间递归（S6）重构为原生运动合成器。通过将预训练主干提取的连续特征表示交错成单一交替序列，该扫描结构促使递归编码每个位置的时间观测于共享隐藏状态中，仅间隔一个衰减步，使帧间转换成为状态动态的固有部分，而非外部计算量。随后，一系列专用对齐与解码操作将此联合编码提炼为显式运动表示，双路径池化机制则通过平衡注意力驱动选择与均匀时间覆盖进行自适应聚合。该框架可无缝对接CNN、Transformer和Mamba主干族，每对特征仅增加约2.1 GFLOPs。在Diving48数据集上，使用图像预训练主干达85.02% Top-1准确率，视频预训练主干处理整段视频仅需一次前向传播即达86.24%，表明结构性诱导的状态空间动态构成运动建模的原理性且通用基础。
原文摘要 · Abstract (English)
Fine-grained action recognition demands temporal reasoning that general-purpose architectures address through different cost-accuracy tradeoffs: 3D dense operators couple computation to the input volume, while difference-based methods approximate motion through rigid, hand-crafted subtraction of uncontextualized features - each reflecting a deliberate design choice with corresponding limitations in expressiveness or flexibility. We present MamBOA, a backbone-agnostic temporal framework built upon a novel interleaved scan structure that recasts the selective state-space recurrence (S6) as a native motion synthesizer. By interleaving consecutive feature representations extracted from a pretrained backbone into a single alternating sequence, the proposed scan structurally drives the recurrence to encode both temporal observations of each position within a shared hidden state, separated by only a single decay step - rendering the inter-frame transition an intrinsic component of the state dynamics rather than an externally computed quantity. A cascade of dedicated alignment and decoding operations then distills this joint encoding into an explicit motion representation, which a dual-path pooling mechanism adaptively aggregates by balancing attention-driven selection with uniform temporal coverage. The framework interfaces seamlessly with CNN, Transformer, and Mamba backbone families, adding only ~2.1 GFLOPs per feature pair. On Diving48, MamBOA achieves 85.02% Top-1 accuracy with an image-pretrained backbone and 86.24% with a video-pretrained backbone processing the entire video in a single forward pass - demonstrating that structurally induced state-space dynamics constitute a principled and general foundation for motion modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。