arXiv:2507.18276cs.ROcs.CV2025-07ICCV被引 8

用基础模型分部件并推理,让机器人自适应操作新铰链物品

Adaptive Articulated Object Manipulation On The Fly with Foundation Model Reasoning and Part Grounding

  • 基于部件级视觉理解提升泛化能力
  • 在模拟与真实场景中成功操作未见过的铰链物体
  • 适合做具身智能与通用机械操作的研究者

铰链类物体对机器人操作带来多样挑战。由于内部结构不可见,机器人需自适应探索并调整动作以生成成功轨迹。现有方法虽尝试跨类别泛化,但仍面临两大难题:(1) 真实世界铰链物品种类繁多,几何差异大,影响视觉感知与理解;(2) 物体功能与结构差异大,难以建立统一的自适应操作策略。为此,我们提出AdaRPG框架,利用基础模型提取物体部件——其局部几何特征更相似,从而增强功能基元技能的视觉效用泛化能力。为此,我们构建了部件级效用标注数据集用于训练效用模型。此外,AdaRPG利用基础模型中的常识知识,推理复杂机制,生成高层控制代码,调用基元技能函数。仿真与真实实验表明,AdaRPG在新铰链物体类别上具备强大泛化能力。

原文摘要 · Abstract (English)

Articulated objects pose diverse manipulation challenges for robots. Since their internal structures are not directly observable, robots must adaptively explore and refine actions to generate successful manipulation trajectories. While existing works have attempted cross-category generalization in adaptive articulated object manipulation, two major challenges persist: (1) the geometric diversity of real-world articulated objects complicates visual perception and understanding, and (2) variations in object functions and mechanisms hinder the development of a unified adaptive manipulation strategy. To address these challenges, we propose AdaRPG, a novel framework that leverages foundation models to extract object parts, which exhibit greater local geometric similarity than entire objects, thereby enhancing visual affordance generalization for functional primitive skills. To support this, we construct a part-level affordance annotation dataset to train the affordance model. Additionally, AdaRPG utilizes the common knowledge embedded in foundation models to reason about complex mechanisms and generate high-level control codes that invoke primitive skill functions based on part affordance inference. Simulation and real-world experiments demonstrate AdaRPG's strong generalization ability across novel articulated object categories.

机器人操作基础模型部件识别泛化能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。