用状态空间模型提升多任务密集预测的交互能力
Parameter Aware Mamba Model for Multi-task Dense Prediction
- 引入参数感知的Mamba框架，通过双状态空间专家建模任务特性
- 在NYUD-v2和PASCAL-Context上实现更优的多任务性能
- 适合需要高效融合多任务特征的视觉密集预测场景
理解多任务密集预测中任务间的相互关系至关重要。现有方法主要依赖卷积层和注意力机制探索任务级交互。本文提出一种基于解码器的新框架——参数感知的Mamba模型（PAMM），专为多任务学习中的密集预测设计。不同于使用Transformer建模全局任务关系的方法，PAMM利用状态空间模型（S4）丰富的可扩展参数来增强任务连通性。其采用双状态空间参数专家，整合并设定任务特定的参数先验，捕捉各任务内在属性。该方法不仅促进精确的多任务交互，还通过结构化状态空间序列模型实现任务先验的全局融合。此外，我们采用多方向希尔伯特扫描法构建多角度特征序列，提升序列模型对二维数据的感知能力。在NYUD-v2和PASCAL-Context基准上的大量实验验证了所提方法的有效性。
原文摘要 · Abstract (English)
Understanding the inter-relations and interactions between tasks is crucial for multi-task dense prediction. Existing methods predominantly utilize convolutional layers and attention mechanisms to explore task-level interactions. In this work, we introduce a novel decoder-based framework, Parameter Aware Mamba Model (PAMM), specifically designed for dense prediction in multi-task learning setting. Distinct from approaches that employ Transformers to model holistic task relationships, PAMM leverages the rich, scalable parameters of state space models to enhance task interconnectivity. It features dual state space parameter experts that integrate and set task-specific parameter priors, capturing the intrinsic properties of each task. This approach not only facilitates precise multi-task interactions but also allows for the global integration of task priors through the structured state space sequence model (S4). Furthermore, we employ the Multi-Directional Hilbert Scanning method to construct multi-angle feature sequences, thereby enhancing the sequence model's perceptual capabilities for 2D data. Extensive experiments on the NYUD-v2 and PASCAL-Context benchmarks demonstrate the effectiveness of our proposed method. Our code is available at https://github.com/CQC-gogopro/PAMM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。