用线性注意力实现高效多任务像素预测，参数少效果好
MTLSI-Net: A Linear Semantic Interaction Network for Parameter-Efficient Multi-Task Dense Prediction
- 通过线性注意力机制捕捉跨任务全局交互，复杂度从二次降至线性
- 在NYUDv2和PASCAL-Context上达到最优性能，参数量显著减少
- 适合需要高效多任务推理的视觉系统，如自动驾驶感知
多任务密集预测旨在同时完成多个像素级任务。然而，由于标准自注意力在高分辨率特征上的二次复杂度，捕捉全局跨任务交互仍具挑战。为此，我们提出多任务线性语义交互网络（MTLSI-Net），通过线性注意力实现跨任务交互。具体包括：多任务多尺度查询线性融合模块，利用共享全局上下文矩阵以线性复杂度捕捉多尺度跨任务依赖；语义令牌蒸馏器，将冗余特征压缩为紧凑语义令牌，提炼关键跨任务知识；跨窗口集成注意力块，通过双分支结构将全局语义注入局部特征，兼顾全局一致性与空间精度。这些组件共同实现线性复杂度下的全面跨任务交互，且参数更少。在NYUDv2和PASCAL-Context上的大量实验表明，MTLSI-Net达到当前最优性能，验证了其在多任务学习中的有效性与高效性。
原文摘要 · Abstract (English)
Multi-task dense prediction aims to perform multiple pixel-level tasks simultaneously. However, capturing global cross-task interactions remains non-trivial due to the quadratic complexity of standard self-attention on high-resolution features. To address this limitation, we propose a Multi-Task Linear Semantic Interaction Network (MTLSI-Net), which facilitates cross-task interaction through linear attention. Specifically, MTLSI-Net incorporates three key components: a Multi-Task Multi-scale Query Linear Fusion Block, which captures cross-task dependencies across multiple scales with linear complexity using a shared global context matrix; a Semantic Token Distiller that compresses redundant features into compact semantic tokens, distilling essential cross-task knowledge; and a Cross-Window Integrated attention Block that injects global semantics into local features via a dual-branch architecture, preserving both global consistency and spatial precision. These components collectively enable the network to capture comprehensive cross-task interactions at linear complexity with reduced parameters. Extensive experiments on NYUDv2 and PASCAL-Context demonstrate that MTLSI-Net achieves state-of-the-art performance, validating its effectiveness and efficiency in multi-task learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。