用LoRA适配DINOv2提升多摄像头鸟瞰图分割的鲁棒性
Robust Bird's Eye View Segmentation by Adapting DINOv2
- 用低秩适配（LoRA）改造DINOv2，使其适应鸟瞰图任务
- 在亮度、天气等干扰下表现更稳定，模型越大越明显
- 参数少、训练快，适合资源受限的自动驾驶系统
从多摄像头图像中提取鸟瞰图（BEV）表示，为自动驾驶提供一种低成本、可扩展的替代方案，无需依赖激光雷达。然而，现有BEV方法在亮度变化、天气恶劣或相机故障等退化条件下性能显著下降。为提升BEV感知的鲁棒性，本文提出使用低秩适配（LoRA）将大型视觉基础模型DINOv2适配至BEV估计任务。该方法基于DINOv2强大的表征空间，在当前领先的SimpleBEV框架中实现适配。实验表明，该方法在多种退化条件下均提升了BEV感知的鲁棒性，且随着模型规模和输入分辨率的增加，增益更加显著。此外，适配后的表示在参数量更少、训练收敛更快方面也展现出优越性。
原文摘要 · Abstract (English)
Extracting a Bird's Eye View (BEV) representation from multiple camera images offers a cost-effective, scalable alternative to LIDAR-based solutions in autonomous driving. However, the performance of the existing BEV methods drops significantly under various corruptions such as brightness and weather changes or camera failures. To improve the robustness of BEV perception, we propose to adapt a large vision foundational model, DINOv2, to BEV estimation using Low Rank Adaptation (LoRA). Our approach builds on the strong representation space of DINOv2 by adapting it to the BEV task in a state-of-the-art framework, SimpleBEV. Our experiments show increased robustness of BEV perception under various corruptions, with increasing gains from scaling up the model and the input resolution. We also showcase the effectiveness of the adapted representations in terms of fewer learnable parameters and faster convergence during training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。