arXiv:2503.07046cs.CV2025-03被引 3

用Mamba架构做光流估计,速度快精度高。

MambaFlow: A Mamba-Centric Architecture for End-to-End Optical Flow Estimation

  • 双Mamba结构捕捉局部关联与跨模态交互
  • 在Sintel上精度超过SEA-RAFT,适合低资源设备
  • 自适应特征融合+递归解码,兼顾效率与全局信息

近期,Mamba架构在图像分类和分割等视觉任务中表现卓越,但在光流估计领域尚未应用。本文提出MambaFlow,首个以Mamba为核心设计的端到端光流估计框架,兼具高精度与高效性。其包含两个核心组件:(1) PolyMamba,通过自注意力式与跨模态式双Mamba模块,实现深层上下文建模与有效特征融合;(2) PulseMamba,采用注意力引导聚合器(AGA)动态加权融合特征,并利用Mamba内在的循环机制进行自回归光流解码,实现高效信息传播。大量实验表明,MambaFlow在基准数据集上性能媲美主流方法。相比SEA-RAFT,其在Sintel数据集上取得更高精度,展现出在资源受限设备上的实际部署潜力。代码将在论文接收后公开。

原文摘要 · Abstract (English)

Recently, the Mamba architecture has demonstrated significant successes in various computer vision tasks, such as classification and segmentation. However, its application to optical flow estimation remains unexplored. In this paper, we introduce MambaFlow, a novel framework designed to leverage the high accuracy and efficiency of the Mamba architecture for capturing locally correlated features while preserving global information in end-to-end optical flow estimation. To our knowledge, MambaFlow is the first architecture centered around the Mamba design tailored specifically for optical flow estimation. It comprises two key components: (1) PolyMamba, which enhances feature representation through a dual-Mamba architecture, incorporating a Self-Mamba module for intra-token modeling and a Cross-Mamba module for inter-modality interaction, enabling both deep contextualization and effective feature fusion; and (2) PulseMamba, which leverages an Attention Guidance Aggregator (AGA) to adaptively integrate features with dynamically learned weights in contrast to naive concatenation, and then employs the intrinsic recurrent mechanism of Mamba to perform autoregressive flow decoding, facilitating efficient flow information dissemination. Extensive experiments demonstrate that MambaFlow achieves remarkable results comparable to mainstream methods on benchmark datasets. Compared to SEA-RAFT, MambaFlow attains higher accuracy on the Sintel benchmark, demonstrating stronger potential for real-world deployment on resource-constrained devices. The source code will be made publicly available upon acceptance of the paper.

光流估计Mamba架构端到端高效模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。