用几何引导的自蒸馏,让弱监督下跨视角定位更准更快。
GeoDistill: Geometry-Guided Self-Distillation for Weakly Supervised Cross-View Localization
- 通过视野遮掩生成学生模型输入,教师模型指导学生聚焦关键特征。
- 在无精确位置标注下仍实现高精度定位,误差降低23%以上。
- 适合自动驾驶、增强现实等需要大规模室外定位的场景。
跨视角定位任务旨在通过匹配地面图像与卫星图像来估计相机的3自由度(3-DoF)位姿,对自动驾驶和增强现实等大规模户外应用至关重要。现有方法多依赖昂贵的真值位姿标注进行全监督学习。本文提出GeoDistill——一种几何引导的弱监督自蒸馏框架,采用教师-学生架构,结合视场(FoV)掩码机制,提升局部特征学习能力。教师模型处理全景图像,学生模型则基于视场遮掩生成的有限视场图像预测位置。通过将学生预测与教师对齐,学生能聚焦车道线等关键特征,忽略道路等无纹理区域,从而提升定位精度并降低不确定性,无论查询图像是全景还是有限视场。实验表明,GeoDistill在多种框架下显著提升性能。此外,我们引入一种新型朝向估计网络,可在无需精确平面位置标注的情况下预测相对朝向。该方法为真实世界跨视角定位提供了可扩展、高效的解决方案。代码与模型见https://github.com/tongshw/GeoDistill。
原文摘要 · Abstract (English)
Cross-view localization, the task of estimating a camera's 3-degrees-of-freedom (3-DoF) pose by aligning ground-level images with satellite images, is crucial for large-scale outdoor applications like autonomous navigation and augmented reality. Existing methods often rely on fully supervised learning, which requires costly ground-truth pose annotations. In this work, we propose GeoDistill, a Geometry guided weakly supervised self distillation framework that uses teacher-student learning with Field-of-View (FoV)-based masking to enhance local feature learning for robust cross-view localization. In GeoDistill, the teacher model localizes a panoramic image, while the student model predicts locations from a limited FoV counterpart created by FoV-based masking. By aligning the student's predictions with those of the teacher, the student focuses on key features like lane lines and ignores textureless regions, such as roads. This results in more accurate predictions and reduced uncertainty, regardless of whether the query images are panoramas or limited FoV images. Our experiments show that GeoDistill significantly improves localization performance across different frameworks. Additionally, we introduce a novel orientation estimation network that predicts relative orientation without requiring precise planar position ground truth. GeoDistill provides a scalable and efficient solution for real-world cross-view localization challenges. Code and model can be found at https://github.com/tongshw/GeoDistill.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。