让机器人在杂乱中自动推理障碍物清除顺序,实现精准抓取。
Obstruction reasoning for robotic grasping
- 基于目标物体的遮挡路径进行多步推理,每步结合遮挡视觉线索。
- 在10万+条人工标注的遮挡路径上训练,真实世界抓取成功率显著提升。
- 适合需要复杂环境操作的机器人研发人员,尤其关注抓取规划者。
在杂乱环境中成功抓取不仅需要模型视觉定位目标物体,还需推理必须清除的障碍物。现有视觉语言具身推理模型虽具备空间理解能力,但在遮挡推理和可达性规划方面仍受限。为此,我们提出UNOGrasp——一种基于学习的视觉语言模型,可进行视觉定位的遮挡推理,推断清除障碍物以抓取目标所需的动作序列。我们设计了一种基于目标物体产生的遮挡路径的多步推理流程,并通过遮挡感知的视觉线索锚定每一步推理,激励模型推理能力。UNOGrasp结合监督微调与可验证推理奖励的强化学习微调。此外,我们构建了UNOBench,一个大规模数据集,基于MetaGraspNetV2,包含超过10万条由人工标注的遮挡路径,涵盖遮挡率、接触点及自然语言指令。大量实验与真实机器人测试表明,UNOGrasp在合成与真实环境中均显著提升遮挡推理与抓取成功率,优于通用及专有方案。项目网站:https://tev-fbk.github.io/UnoGrasp/
原文摘要 · Abstract (English)
Successful robotic grasping in cluttered environments not only requires a model to visually ground a target object but also to reason about obstructions that must be cleared beforehand. While current vision-language embodied reasoning models show emergent spatial understanding, they remain limited in terms of obstruction reasoning and accessibility planning. To bridge this gap, we present UNOGrasp, a learning-based vision-language model capable of performing visually-grounded obstruction reasoning to infer the sequence of actions needed to unobstruct the path and grasp the target object. We devise a novel multi-step reasoning process based on obstruction paths originated by the target object. We anchor each reasoning step with obstruction-aware visual cues to incentivize reasoning capability. UNOGrasp combines supervised and reinforcement finetuning through verifiable reasoning rewards. Moreover, we construct UNOBench, a large-scale dataset for both training and benchmarking, based on MetaGraspNetV2, with over 100k obstruction paths annotated by humans with obstruction ratios, contact points, and natural-language instructions. Extensive experiments and real-robot evaluations show that UNOGrasp significantly improves obstruction reasoning and grasp success across both synthetic and real-world environments, outperforming generalist and proprietary alternatives. Project website: https://tev-fbk.github.io/UnoGrasp/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。