arXiv:2510.23119cs.RO2025-10被引 8

用大模型和力反馈实现通用灵巧抓取,支持多种指令和物体。

OmniDexGrasp: Generalizable Dexterous Grasping via Foundation Model and Force Feedback

  • 结合大模型生成人类抓取图像,提升指令泛化能力。
  • 将人手动作转为机器人可执行指令,适配多种灵巧手。
  • 引入力觉反馈自适应调整抓取,确保稳定可靠。

让机器人根据人类指令完成灵巧抓取是机器人领域的重要方向。然而,现有方法因语义灵巧抓取数据集规模有限,难以在不同物体或任务间泛化。大模型虽能增强泛化能力,但其抽象知识与物理执行之间存在鸿沟。为此,我们提出OmniDexGrasp框架,通过融合大模型与迁移控制策略,实现用户指令、灵巧体态和抓取任务的全方位泛化。该框架包含三个核心模块:(i) 利用大模型生成支持多指令、多任务的人类抓取图像,提升泛化能力;(ii) 设计人像到机器人动作的迁移策略,将人类示范转化为可执行动作,实现多形态灵巧手适配;(iii) 采用力觉感知的自适应抓取策略,保障抓取过程鲁棒稳定。仿真与真实机器人实验验证了该框架在多样化用户指令、抓取任务及灵巧手上的有效性,且结果表明其可扩展至灵巧操作任务。

原文摘要 · Abstract (English)

Enabling robots to dexterously grasp and manipulate objects based on human commands is a promising direction in robotics. However, existing approaches are challenging to generalize across diverse objects or tasks due to the limited scale of semantic dexterous grasp datasets. Foundation models offer a new way to enhance generalization, yet directly leveraging them to generate feasible robotic actions remains challenging due to the gap between abstract model knowledge and physical robot execution. To address these challenges, we propose OmniDexGrasp, a generalizable framework that achieves omni-capabilities in user prompting, dexterous embodiment, and grasping tasks by combining foundation models with the transfer and control strategies. OmniDexGrasp integrates three key modules: (i) foundation models are used to enhance generalization by generating human grasp images supporting omni-capability of user prompt and task; (ii) a human-image-to-robot-action transfer strategy converts human demonstrations into executable robot actions, enabling omni dexterous embodiment; (iii) force-aware adaptive grasp strategy ensures robust and stable grasp execution. Experiments in simulation and on real robots validate the effectiveness of OmniDexGrasp on diverse user prompts, grasp task and dexterous hands, and further results show its extensibility to dexterous manipulation tasks.

灵巧抓取大模型力反馈机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。