arXiv:2508.20376cs.CV2025-08ICCV

提出双向交互Mamba,高效实现多任务密集预测中的跨任务信息融合

Enhancing Mamba Decoder with Bidirectional Interaction in Multi-Task Dense Prediction

  • 设计双向扫描机制,统一线性复杂度下融合任务与位置信息
  • 在NYUD-V2和PASCAL-Context上优于现有最先进方法
  • 适合需要高效多任务推理的视觉密集预测场景

充分的跨任务交互对多任务密集预测的成功至关重要,但通常导致高计算开销,使现有方法面临交互完整性与计算效率的权衡。为此,本文提出双向交互Mamba(BIM),引入新型扫描机制,将Mamba建模方法适配于多任务密集预测。一方面,提出双向交互扫描(BI-Scan)机制,在交互过程中构建任务特定的双向序列;通过在统一线性复杂度架构中整合任务优先与位置优先扫描模式,有效保留关键跨任务信息。另一方面,采用多尺度扫描(MS-Scan)机制实现多粒度场景建模,不仅满足不同任务的粒度需求,还增强细粒度的跨任务特征交互。在两个挑战性基准数据集NYUD-V2和PASCAL-Context上的大量实验表明,BIM显著优于现有最先进方法。

原文摘要 · Abstract (English)

Sufficient cross-task interaction is crucial for success in multi-task dense prediction. However, sufficient interaction often results in high computational complexity, forcing existing methods to face the trade-off between interaction completeness and computational efficiency. To address this limitation, this work proposes a Bidirectional Interaction Mamba (BIM), which incorporates novel scanning mechanisms to adapt the Mamba modeling approach for multi-task dense prediction. On the one hand, we introduce a novel Bidirectional Interaction Scan (BI-Scan) mechanism, which constructs task-specific representations as bidirectional sequences during interaction. By integrating task-first and position-first scanning modes within a unified linear complexity architecture, BI-Scan efficiently preserves critical cross-task information. On the other hand, we employ a Multi-Scale Scan~(MS-Scan) mechanism to achieve multi-granularity scene modeling. This design not only meets the diverse granularity requirements of various tasks but also enhances nuanced cross-task feature interactions. Extensive experiments on two challenging benchmarks, \emph{i.e.}, NYUD-V2 and PASCAL-Context, show the superiority of our BIM vs its state-of-the-art competitors.

多任务学习密集预测Mamba高效建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。