用少量真实数据让合成数据训练的分割模型更好适应真实场景。
RAFT -- A Domain Adaptation Framework for RGB & LiDAR Semantic Segmentation
- 通过数据增强与主动学习，仅需少量真实标签即可实现域适应。
- 在SYNTHIA→Cityscapes等任务上，mIoU提升2.1%至79.9%。
- 适合缺乏标注数据但有合成数据的自动驾驶视觉系统部署。
图像分割是场景理解的重要计算机视觉技术，但实际部署受限于高质量、精细标注数据集的需求。合成数据可提供高质量标签，减少人工标注负担，但基于合成数据训练的深度神经网络常面临‘合成到真实’（Syn2Real）性能下降问题。为此，本文提出RAFT框架，通过最小量的真实标注数据，结合数据与特征增强及主动学习，实现图像分割模型的域适应。在合成到真实的基准测试SYNTHIA→Cityscapes和GTAV→Cityscapes上，RAFT超越现有最优方法HALO：前者mIoU提升2.1%至79.9%，后者提升0.4%至78.2%。在真实到真实的Cityscapes→ACDC基准上也优于HALO，mIoU提升1.3%至73.2%。此外，我们分析了标注预算和RAFT各组件对迁移性能的影响。
原文摘要 · Abstract (English)
Image segmentation is a powerful computer vision technique for scene understanding. However, real-world deployment is stymied by the need for high-quality, meticulously labeled datasets. Synthetic data provides high-quality labels while reducing the need for manual data collection and annotation. However, deep neural networks trained on synthetic data often face the Syn2Real problem, leading to poor performance in real-world deployments. To mitigate the aforementioned gap in image segmentation, we propose RAFT, a novel framework for adapting image segmentation models using minimal labeled real-world data through data and feature augmentations, as well as active learning. To validate RAFT, we perform experiments on the synthetic-to-real "SYNTHIA->Cityscapes" and "GTAV->Cityscapes" benchmarks. We managed to surpass the previous state of the art, HALO. SYNTHIA->Cityscapes experiences an improvement in mIoU* upon domain adaptation of 2.1%/79.9%, and GTAV->Cityscapes experiences a 0.4%/78.2% improvement in mIoU. Furthermore, we test our approach on the real-to-real benchmark of "Cityscapes->ACDC", and again surpass HALO, with a gain in mIoU upon adaptation of 1.3%/73.2%. Finally, we examine the effect of the allocated annotation budget and various components of RAFT upon the final transfer mIoU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。