arXiv:2603.07866cs.ROcs.LG2026-03

让四足机器人在看不清物体时,靠语言指令精准抓取工具。

Language-Guided Grasping under Partial Observation for Mobile Manipulation in Field Inspection and Maintenance

  • 用语言描述定位目标,结合视觉和深度信息重建物体形状
  • 在杂乱场景中实现9/10成功抓取,远超传统方法的3/10
  • 适合需要远程操作、高危环境下的巡检与维护任务

海上巡检与维护正越来越多地使用腿式机器人进行常规感知,但许多有用的操作仍需与工具、容器及任务相关物体进行物理交互。让机器人执行这些任务可减少操作员在密闭、高处或可能爆炸区域的暴露风险。本文提出一种针对腿式移动机械臂在部分观测条件下的语言引导抓取流程。操作员指定目标后,系统通过开放词汇检测与可提示分割将目标在RGB图像中定位,提取以物体为中心的RGB-D点云,利用深度补偿与点云补全提升稀疏几何精度,并基于碰撞、间隙、可达性及接近约束选择6自由度抓取姿态。该系统部署于配备机械臂的四足机器人,在两个模拟巡检与维护中小型物品检索的杂乱桌面场景中进行评估。在成对试验中,所提流程实现9/10次成功抓取,而视依赖基线仅实现3/10次。在此受控环境中,物体中心补全与执行感知选择有效降低了接近过程中的碰撞,提升了语言引导抓取在监督式现场操作中的可靠性。

原文摘要 · Abstract (English)

Offshore inspection and maintenance have increasingly been using legged robots for routine sensing, yet many useful interventions still require physical interaction with tools, containers, and task-relevant objects. Employing robots for these tasks can reduce operators' exposure in confined, elevated, or potentially explosive areas. This paper presents a language-guided grasping pipeline for a legged mobile manipulator operating under partial observation. An operator defines the target, the system grounds it in RGB with open-vocabulary detection and promptable segmentation, extracts an object-centric RGB-D point cloud, improves sparse geometry through depth compensation and point-cloud completion, and selects a 6-DoF grasp using collision, clearance, reachability, and approach constraints. The system is implemented on a quadruped robot with an arm and evaluated in two cluttered tabletop scenes motivated by small-object retrieval during inspection and maintenance. Across paired trials, the proposed pipeline achieved 9/10 successful grasps, compared with 3/10 for a view-dependent deployment baseline. In this controlled setting, object-centric completion and execution-aware selection reduced approach collisions and improved the reliability of language-guided grasping for supervised field manipulation.

机器人抓取语言引导四足机器人现场运维

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。