用视觉语言模型直接感知环境,让机器人在复杂场景中触碰障碍物仍能高效导航。
Direct Contact-Tolerant Motion Planning With Vision Language Models
- 通过视觉语言模型直接分析图像,生成带接触感知的点云
- 在真实车形机器人上实现稳定导航,性能优于多个基线方法
- 适合需要灵活应对移动/可变形障碍物的智能导航场景
在杂乱环境中导航时常需容忍与可移动或可变形物体的接触以保持效率。现有接触容忍运动规划(CTMP)方法依赖预构建地图或障碍物集合等间接空间表征,导致精度不足且难以适应环境不确定性。为此,我们提出直接接触容忍(DCT)规划器,将视觉语言模型(VLM)融入直接点云感知与导航,包含两个关键组件:一是视觉语言模型点云分割器(VPP),利用VLM在图像空间进行接触容忍推理,缓存推理掩码,通过里程计跨帧传播并投影至当前扫描,生成接触感知点云;二是VPP引导导航(VGN),将CTMP建模为在直接接触感知点云约束下的感知-控制优化问题,并由专用深度神经网络(DNN)求解。我们在Isaac Sim和真实车形机器人上实现DCT,验证其在含可移动障碍物的复杂环境中具备鲁棒高效导航能力,各项指标均优于代表性基线。代码已公开:https://github.com/ChrisLeeUM/DCT。
原文摘要 · Abstract (English)
Navigation in cluttered environments often requires robots to tolerate contact with movable or deformable objects to maintain efficiency. Existing contact-tolerant motion planning (CTMP) methods rely on indirect spatial representations (e.g., prebuilt map, obstacle set), resulting in inaccuracies and a lack of adaptiveness to environmental uncertainties. To address this issue, we propose a direct contact-tolerant (DCT) planner, which integrates vision-language models (VLMs) into direct point perception and navigation, including two key components. The first one is VLM point cloud partitioner (VPP), which performs contact-tolerance reasoning in image space using VLM, caches inference masks, propagates them across frames using odometry, and projects them onto the current scan to generate a contact-aware point cloud. The second innovation is VPP guided navigation (VGN), which formulates CTMP as a perception-to-control optimization problem under direct contact-aware point cloud constraints, which is further solved by a specialized deep neural network (DNN). We implement DCT in Isaac Sim and a real car-like robot, demonstrating that DCT achieves robust and efficient navigation in cluttered environments with movable obstacles, outperforming representative baselines across diverse metrics. The code is available at: https://github.com/ChrisLeeUM/DCT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。