用视觉基础模型做教师,让多任务模型高效适应新环境。
Efficient Domain-Adaptive Multi-Task Dense Prediction with Vision Foundation Models
- 用基础模型生成伪标签,自训练提升跨域性能
- 轻量模型比基础模型小10倍以上,仍达顶尖精度
- 适合算力受限的机器人应用,尤其夜间场景
多任务密集预测对机器人至关重要,但部署时面临域偏移问题。现有无监督域适应(UDA)方法多依赖对抗学习,效果不如自训练。本文提出FAMDA框架,利用视觉基础模型(VFMs)作为教师,在自训练中生成目标域高质量伪标签,将基础模型的强泛化能力蒸馏到轻量学生网络。实验表明,FAMDA在标准合成到真实及昼夜适应任务上均达当前最优表现;其轻量版本模型规模小于基础模型10倍,仍保持顶尖精度,适合资源受限的机器人场景。
原文摘要 · Abstract (English)
Multi-task dense prediction, which aims to jointly solve tasks like semantic segmentation and depth estimation, is crucial for robotics applications but suffers from domain shift when deploying models in new environments. While unsupervised domain adaptation (UDA) addresses this challenge for single tasks, existing multi-task UDA methods primarily rely on adversarial learning approaches that are less effective than recent self-training techniques. In this paper, we introduce FAMDA, a simple yet effective UDA framework that addresses this limitation by leveraging Vision Foundation Models (VFMs) as powerful teachers within a self-training paradigm. Our approach integrates Segmentation and Depth foundation models into a self-training paradigm to generate high-quality pseudo-labels for the target domain, effectively distilling their robust generalization capabilities into a single, efficient student network. Extensive experiments show that FAMDA achieves state-of-the-art (SOTA) performance on standard synthetic-to-real UDA multi-task learning (MTL) benchmarks and a challenging new day-to-night adaptation task. Our framework enables the training of highly efficient models; a lightweight variant achieves SOTA accuracy while being more than 10X smaller than foundation models, highlighting FAMDA's suitability for creating domain-adaptive and efficient models for resource-constrained robotics applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。