用大模型DINOv2做视觉里程计,提升鲁棒性和泛化能力。
DINO-VO: A Feature-based Visual Odometry Leveraging a Visual Foundation Model
- 基于DINOv2提取特征,设计专用关键点检测器应对粗粒度问题。
- 融合语义与几何特征,提升定位精度,在多个数据集上表现优异。
- 实时高效,72帧/秒,适合机器人与自动驾驶场景使用。
基于学习的单目视觉里程计在机器人应用中面临鲁棒性、泛化性和效率挑战。近年来,DINOv2等视觉基础模型在多种视觉任务中提升了鲁棒性与泛化能力,但其在视觉里程计中的应用受限于特征粒度粗糙。本文提出DINO-VO,一种基于DINOv2的特征型视觉里程计系统,用于稀疏特征匹配。为解决集成难题,我们设计了针对DINOv2粗粒度特征的关键点检测器;同时,融合鲁棒语义特征与细粒度几何特征,增强局部可定位性。此外,采用Transformer匹配器与可微姿态估计层,实现精准相机运动估计。相比SuperPoint等检测-描述网络,DINO-VO在复杂环境中更具鲁棒性;所提特征描述子优于独立DINOv2粗特征。在TartanAir和KITTI数据集上超越现有帧对帧方法,欧雷克(EuRoC)上表现具竞争力,且在单个GPU上以72 FPS运行,内存占用低于1GB。在室外驾驶场景中,性能媲美视觉SLAM系统,展现良好泛化能力。
原文摘要 · Abstract (English)
Learning-based monocular visual odometry (VO) poses robustness, generalization, and efficiency challenges in robotics. Recent advances in visual foundation models, such as DINOv2, have improved robustness and generalization in various vision tasks, yet their integration in VO remains limited due to coarse feature granularity. In this paper, we present DINO-VO, a feature-based VO system leveraging DINOv2 visual foundation model for its sparse feature matching. To address the integration challenge, we propose a salient keypoints detector tailored to DINOv2's coarse features. Furthermore, we complement DINOv2's robust-semantic features with fine-grained geometric features, resulting in more localizable representations. Finally, a transformer-based matcher and differentiable pose estimation layer enable precise camera motion estimation by learning good matches. Against prior detector-descriptor networks like SuperPoint, DINO-VO demonstrates greater robustness in challenging environments. Furthermore, we show superior accuracy and generalization of the proposed feature descriptors against standalone DINOv2 coarse features. DINO-VO outperforms prior frame-to-frame VO methods on the TartanAir and KITTI datasets and is competitive on EuRoC dataset, while running efficiently at 72 FPS with less than 1GB of memory usage on a single GPU. Moreover, it performs competitively against Visual SLAM systems on outdoor driving scenarios, showcasing its generalization capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。