提出非因果双曲形Mamba，让图像全局交互无方向偏见
Vision Non-Causal Trapezoidal Mamba: Eliminating Directional Scanning in Vision SSMs with Second-Order Dynamics
- 采用二阶非因果状态空间建模，实现图像令牌全连接
- 旋转翻转下性能下降减少，边界交并比提升3.7点
- 适合追求高效鲁棒视觉识别的开发者使用
视觉状态空间模型（SSMs）作为视觉Transformer的替代方案，但多数仍继承因果序列建模带来的方向性扫描问题。这种扫描方式在序列数据中有效，却引入空间偏见和方向敏感表示。本文提出视觉非因果双曲形Mamba（VNCT），一种基于二阶非因果状态空间的视觉模型，可在单次遍历中实现所有图像令牌的交互，消除方向性扫描，显著降低单图推理延迟。该模型表现出更强的方向鲁棒性，在图像旋转与翻转时性能下降更小，边界交并比（Boundary IoU）最高提升3.7点，显著改善边界保留与目标定位精度。在ImageNet-1K分类、COCO目标检测与实例分割、ADE20K语义分割任务上，VNCT持续优于具有方向扫描的视觉SSMs及一阶非因果SSMs。结果表明，高性能视觉SSMs无需依赖方向性扫描，而二阶非因果状态空间建模提供了一种简单、高效且鲁棒的视觉识别新范式。
原文摘要 · Abstract (English)
State Space Models (SSMs) have emerged as an alternative to Vision Transformers, yet most vision SSMs inherit directional token scanning from causal sequence modeling. While effective for sequential data, directional scanning introduces spatial bias and orientation-sensitive representations. We present Vision Non-Causal Trapezoidal Mamba (VNCT), a second-order non-causal vision SSM that enables all image tokens to interact in a single pass, eliminating direSctional scanning and achieving low single-image inference latency. VNCT exhibits more orientation-robust representations, showing reduced performance degradation under image rotations and flips, while improving Boundary IoU by up to 3.7 points, leading to more accurate boundary preservation and object localization. Across ImageNet-1K classification, COCO object detection and instance segmentation, and ADE20K semantic segmentation, VNCT consistently outperforms both directional-scanning vision SSMs and first-order non-causal SSMs. These results show that directional scanning is unnecessary for high-performance vision SSMs and that second-order non-causal state-space modeling offers a simple, efficient, and robust alternative for visual recognition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。