用神经内部模型让机械手高效完成复杂操作，像人一样自然。
MoDex: Planning High-Dimensional Dexterous Control via Learning Neural Internal Models
- 通过神经网络学习手部动力学特性，构建可训练的内部模型。
- 在仿真与真实世界中实现高维操作任务的高效规划与快速迁移。
- 支持与大语言模型结合生成手势，适合机器人灵巧操控研究者。
高维动作空间中的手部控制长期面临挑战，而人类却能轻松完成灵巧操作。本文受人类行为中内部模型概念启发，将灵巧手视为可学习系统，提出MoDex框架：包含捕捉手部动力学特性的神经网络及双向规划方法，兼具训练与规划效率。为验证其通用性，进一步集成外部模型以实现物体在手中操作，并结合大语言模型（LLM）在仿真与真实世界中生成多种手势。在多种灵巧手上开展的大量实验表明，该方法具备良好的任务学习数据效率及跨任务迁移能力。
原文摘要 · Abstract (English)
Controlling hands in high-dimensional action space has been a longstanding challenge, yet humans naturally perform dexterous tasks with ease. In this paper, we draw inspiration from the concept of internal model exhibited in human behavior and reconsider dexterous hands as learnable systems. Specifically, we introduce MoDex, a framework that includes a couple of neural networks (NNs) capturing the dynamical characteristics of hands and a bidirectional planning approach, which demonstrates both training and planning efficiency. To show the versatility of MoDex, we further integrate it with an external model to manipulate in-hand objects and a large language model (LLM) to generate various gestures in both simulation and real world. Extensive experiments on different dexterous hands address the data efficiency in learning a new task and the transferability between different tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。