轻量级多尺度融合框架，提升视觉模型在多任务感知中的效率与精度。
DPNeXt: A Lightweight Multi-Scale Feature Fusion Framework for Efficient ViT-Based Multi-Task Dense Prediction
- 采用双深度可分离倒置瓶颈结构，实现聚焦融合的解码与任务独立化设计。
- 在Cityscapes和NYUv2上均超越现有SOTA，参数量减少78.6%且推理更快。
- 无需额外标注或计算开销，通过对称边界监督增强几何一致性，适合边缘设备部署。
机器人感知系统中的多任务学习（MTL）通过融合语义分割与深度估计，支持全面的3D场景理解。尽管视觉基础模型（VFMs）被广泛用作鲁棒特征编码器，现有解码策略仍构成关键瓶颈。为此，我们提出DPNeXt，一种轻量级多尺度特征融合解码器，作为标准密集预测变压器（DPT）的高效替代方案。DPNeXt利用双深度可分离倒置瓶颈，通过融合导向解码与任务独立模块化，提升冻结VFM的利用率。为缓解任务间的负向归纳转移，引入多任务边界引导（MTBG）策略：不同于以往添加融合模块或门控机制的方法，MTBG施加对称边界监督，促进几何一致性，且不增加标注或推理成本。在Cityscapes上，DPNeXt-S超越现有SOTA；DPNeXt-B进一步提升整体性能，达到对比方法中的最佳结果。在NYUv2上，DPNeXt-B同样取得最优的语义分割与深度估计效果，同时所需可训练参数远少于先前大规模MTL模型。相较于标准DPT，DPNeXt-S参数量减少78.6%，在资源受限笔记本硬件上推理速度最快。代码、模型检查点及演示视频将发布于https://github.com/kangjehun/DPNeXt。
原文摘要 · Abstract (English)
Multi-Task Learning (MTL) in robotics perception systems supports comprehensive 3D spatial scene understanding by integrating semantic segmentation and depth estimation. While Vision Foundation Models (VFMs) are increasingly adopted as robust feature encoders, existing decoding strategies present a critical bottleneck. To address this, we propose DPNeXt, a streamlined multi-scale feature fusion decoder and efficient alternative to the standard Dense Prediction Transformer (DPT). DPNeXt uses dual depthwise separable inverted bottlenecks to improve frozen VFM utilization through fusion-centric decoding and independent task modularization. To further mitigate negative inductive transfer between tasks, we introduce the Multi-Task Boundary Guidance (MTBG) strategy. Unlike prior boundary-aware methods that add fusion modules or gating, MTBG applies symmetric boundary-focused supervision to encourage geometric consistency without extra annotation or inference cost. Experiments on Cityscapes show that DPNeXt-S outperforms prior state-of-the-art (SOTA) MTL models, while DPNeXt-B further improves the overall performance and achieves the best results among the compared methods. On NYUv2, DPNeXt-B also achieves the best semantic segmentation and depth estimation results among the compared methods while requiring substantially fewer trainable parameters than prior large-scale MTL models. Compared with the standard DPT, DPNeXt-S reduces trainable parameters by 78.6% and achieves the fastest inference speed among the compared models on resource-constrained laptop hardware. The source code, model checkpoints, and a demo video will be made available at https://github.com/kangjehun/DPNeXt.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。