arXiv:2604.01958cs.CV2026-04中稿 · ECCV

通过动态区域稀疏交互,高效融合红外与可见光视频。

MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction

论文配图:MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction
图 1 · 摘自论文原文
  • 根据光流识别动态区域,只在关键处做跨模态注意力
  • 在640×480下达到14.16帧/秒,兼顾速度与质量
  • 适合需要实时融合的无人机、夜视等场景

红外与可见光视频融合旨在结合红外图像的目标显著性与可见光图像的纹理细节,生成语义丰富的融合结果。然而,现有方法多针对静态图像设计,难以有效处理视频中的帧间运动。当前视频融合方法虽通过帧间交互提升时序一致性,但计算开销大。为此,我们提出MAVFusion,一种端到端视频融合框架,采用运动感知稀疏交互机制,在保持优异融合质量的同时显著提升效率。具体地,利用光流识别多模态序列中的动态区域,仅在这些稀疏区域施加高成本的跨模态注意力,以捕捉显著变化并促进模态间信息交换;对于静态背景区域,则使用轻量级弱交互模块维持结构与外观完整性。通过解耦动态与静态区域处理,MAVFusion同时实现时序一致性与细粒度细节保留,并大幅加速推理。大量实验表明,MAVFusion在多个红外-可见光视频基准上达到领先性能,在640×480分辨率下实现14.16 FPS。源代码将发布于https://github.com/ixilai/MAVFusion。

原文摘要 · Abstract (English)

Infrared and visible video fusion combines the object saliency from infrared images with the texture details from visible images to produce semantically rich fusion results. However, most existing methods are designed for static image fusion and cannot effectively handle frame-to-frame motion in videos. Current video fusion methods improve temporal consistency by introducing interactions across frames, but they often require high computational cost. To mitigate these challenges, we propose MAVFusion, an end-to-end video fusion framework featuring a motion-aware sparse interaction mechanism that enhances efficiency while maintaining superior fusion quality. Specifically, we leverage optical flow to identify dynamic regions in multi-modal sequences, adaptively allocating computationally intensive cross-modal attention to these sparse areas to capture salient transitions and facilitate inter-modal information exchange. For static background regions, a lightweight weak interaction module is employed to maintain structural and appearance integrity. By decoupling the processing of dynamic and static regions, MAVFusion simultaneously preserves temporal consistency and fine-grained details while significantly accelerating inference. Extensive experiments demonstrate that MAVFusion achieves state-of-the-art performance on multiple infrared and visible video benchmarks, achieving a speed of 14.16 FPS at $640 \times 480$ resolution. The source code will be available at https://github.com/ixilai/MAVFusion.

视频融合红外可见光稀疏交互实时处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。