用激光雷达辅助训练2D图像模型，提升恶劣天气下的感知鲁棒性。
LiDAR-Anchored Collaborative Distillation for Robust 2D Representations
- 以激光雷达为自监督信号，指导2D图像模型学习更鲁棒的特征
- 在多种恶劣天气下下游任务表现优于现有方法，泛化能力强
- 无需额外标注，适合自动驾驶等真实场景应用
随着深度学习的发展，自监督学习已取得显著进展，使2D图像编码器能够为各类下游任务提取有效特征，尤其适用于基于视觉的系统。然而，预训练的2D图像编码器在噪声干扰和恶劣天气条件下（如雨、雾、夜间）的表现仍不理想，难以满足鲁棒视觉感知需求。为此，我们提出一种新颖的自监督方法——协同蒸馏（Collaborative Distillation），利用3D激光雷达作为自监督信号，增强2D图像编码器在复杂环境中的鲁棒性，同时保持其原有能力。实验表明，该方法在多种下游任务中均优于对比方法，在不同天气与光照条件下表现出强泛化能力。此外，该方法还提升了模型对3D空间信息的感知能力，得益于激光雷达的几何特性。这一进展凸显了方法在真实场景中的实用性与适应性。
原文摘要 · Abstract (English)
As deep learning continues to advance, self-supervised learning has made considerable strides. It allows 2D image encoders to extract useful features for various downstream tasks, including those related to vision-based systems. Nevertheless, pre-trained 2D image encoders fall short in conducting the task under noisy and adverse weather conditions beyond clear daytime scenes, which require for robust visual perception. To address these issues, we propose a novel self-supervised approach, \textbf{Collaborative Distillation}, which leverages 3D LiDAR as self-supervision to improve robustness to noisy and adverse weather conditions in 2D image encoders while retaining their original capabilities. Our method outperforms competing methods in various downstream tasks across diverse conditions and exhibits strong generalization ability. In addition, our method also improves 3D awareness stemming from LiDAR's characteristics. This advancement highlights our method's practicality and adaptability in real-world scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。