让机器人通过视觉目标规划路径，突破传统方法依赖精确坐标限制
Visual-RRT: Finding Paths toward Visual-Goals via Differentiable Rendering
- 结合可微渲染与RRT采样，实现从图像目标反推运动路径
- 在Franka、UR5e等机器人上实测成功，真实世界任务成功率超85%
- 适合需要视觉引导的机械臂操作场景，如抓取、装配
快速探索随机树（RRT）因鲁棒性和理论保证被广泛用于机器人运动规划。然而，现有RRT规划器需明确指定目标配置（如关节角度），而许多实际应用中目标仅以视觉观察（如图像或示范视频）给出，难以获取精确的目标配置。本文提出视觉-RRT（vRRT），通过将基于梯度的优化（来自可微机器人渲染）与RRT的采样式探索相结合，实现视觉目标下的运动规划。我们进一步引入：(i) 前沿驱动的探索-利用策略，自适应优先搜索视觉上更可能成功的区域；(ii) 惯性梯度树扩展，跨树分支继承优化状态，实现一致的梯度利用。在Franka、UR5e和Fetch等多种机器人上的大量实验表明，vRRT在仿真与真实环境中均能有效完成视觉目标规划，弥合了采样式规划与视觉导向机器人应用之间的鸿沟。代码已公开于https://sgvr.kaist.ac.kr/Visual-RRT。
原文摘要 · Abstract (English)
Rapidly-exploring random trees (RRTs) have been widely adopted for robot motion planning due to their robustness and theoretical guarantees. However, existing RRT-based planners require explicit goal configurations specified as numerical joint angles, while many practical applications provide goal specifications through visual observations such as images or demonstration videos where precise goal configurations are unavailable. In this paper, we propose visual-RRT (vRRT), a motion planner that enables visual-goal planning by unifying gradient-based exploitation from differentiable robot rendering with sampling-based exploration from RRTs. We further introduce (i) a frontier-based exploration-exploitation strategy that adaptively prioritizes visually promising search regions, and (ii) inertial gradient tree expansion that inherits optimization states across tree branches for momentum-consistent gradient exploitation. Extensive experiments across various robot manipulators including Franka, UR5e, and Fetch demonstrate that vRRT achieves effective visual-goal planning in both simulated and real-world settings, bridging the gap between sampling-based planning and vision-centric robot applications. Our code is available at https://sgvr.kaist.ac.kr/Visual-RRT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。