用状态空间模型实现高效视频融合，无需光流估计
MambaVF: State Space Model for Efficient Video Fusion
- 将视频融合建模为状态序列更新，线性复杂度捕捉长时依赖
- 轻量级模块替代光流对齐，节省92.25%参数和88.79%计算量
- 适合需要高效率的实时视频处理场景，如医疗与多模态融合
视频融合是多种视频处理任务的基础技术。然而，现有方法严重依赖光流估计与特征扭曲，导致计算开销大且难以扩展。本文提出MambaVF，一种基于状态空间模型（SSMs）的高效视频融合框架，通过隐式时间建模避免显式运动估计。首先，将视频融合重构为序列状态更新过程，以线性复杂度捕获长时依赖，显著降低计算与内存开销。其次，设计了一种轻量级基于SSM的融合模块，采用时空双向扫描机制替代传统光流引导对齐，实现跨帧信息高效聚合。在多个基准测试中，MambaVF在多曝光、多焦点、红外-可见光及医学视频融合任务上均达到领先性能。实验表明，相比现有方法，其参数减少高达92.25%，计算量降低88.79%，速度提升2.1倍。
原文摘要 · Abstract (English)
Video fusion is a fundamental technique in various video processing tasks. However, existing video fusion methods heavily rely on optical flow estimation and feature warping, resulting in severe computational overhead and limited scalability. This paper presents MambaVF, an efficient video fusion framework based on state space models (SSMs) that performs temporal modeling without explicit motion estimation. First, by reformulating video fusion as a sequential state update process, MambaVF captures long-range temporal dependencies with linear complexity while significantly reducing computation and memory costs. Second, MambaVF proposes a lightweight SSM-based fusion module that replaces conventional flow-guided alignment via a spatio-temporal bidirectional scanning mechanism. This module enables efficient information aggregation across frames. Extensive experiments across multiple benchmarks demonstrate that our MambaVF achieves state-of-the-art performance in multi-exposure, multi-focus, infrared-visible, and medical video fusion tasks. We highlight that MambaVF enjoys high efficiency, reducing up to 92.25% of parameters and 88.79% of computational FLOPs and a 2.1x speedup compared to existing methods. Project page: https://mambavf.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。