用单目深度图替代激光雷达,实现无死角导航
Learning Vision-Based Omnidirectional Navigation: A Teacher-Student Approach Using Monocular Depth Estimation
- 教师学生框架:用激光雷达训练,单目深度图部署
- 实测成功率超96%,复杂3D障碍物表现更优
- 全程机载运行,适合工业移动机器人落地
工业场景中可靠避障需三维环境理解,但广泛使用的2D LiDAR仅感知水平切片,遗漏上下方关键障碍。本文提出基于单目深度估计的师生框架,无需激光雷达。教师策略在NVIDIA Isaac Lab中通过近端策略优化(PPO)训练,利用包含完整机器人轮廓的2D LiDAR观测学习鲁棒导航;其行为被蒸馏至仅依赖四路RGB相机经微调的Depth Anything V2模型生成的单目深度图的学生策略。完整推理流程(单目深度估计、策略执行、电机控制)全部在搭载于DJI RoboMaster平台的NVIDIA Jetson Orin AGX上完成,无需外部计算。仿真中,学生成功率82%-96.5%,持续优于标准2D LiDAR教师(50%-89%)。真实实验中,面对悬垂结构、低矮物体等超出2D LiDAR扫描平面的复杂3D障碍物,基于单目深度估计的学生策略表现更优。
原文摘要 · Abstract (English)
Reliable obstacle avoidance in industrial settings demands 3D scene understanding, but widely used 2D LiDAR sensors perceive only a single horizontal slice of the environment, missing critical obstacles above or below the scan plane. We present a teacher-student framework for vision-based mobile robot navigation that eliminates the need for LiDAR sensors. A teacher policy trained via Proximal Policy Optimization (PPO) in NVIDIA Isaac Lab leverages privileged 2D LiDAR observations that account for the full robot footprint to learn robust navigation. The learned behavior is distilled into a student policy that relies solely on monocular depth maps predicted by a fine-tuned Depth Anything V2 model from four RGB cameras. The complete inference pipeline, comprising monocular depth estimation (MDE), policy execution, and motor control, runs entirely onboard an NVIDIA Jetson Orin AGX mounted on a DJI RoboMaster platform, requiring no external computation for inference. In simulation, the student achieves success rates of 82-96.5%, consistently outperforming the standard 2D LiDAR teacher (50-89%). In real-world experiments, the MDE-based student outperforms the 2D LiDAR teacher when navigating around obstacles with complex 3D geometries, such as overhanging structures and low-profile objects, that fall outside the single scan plane of a 2D LiDAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。