基于神经科学设计分步抓取策略，提升机器人灵巧操作能力。
Modality-Driven Design for Multi-Step Dexterous Manipulation: Insights from Neuroscience
- 按感官主导模式分解抓取任务为三步子技能
- 真实机器人测试中成功完成抓取旋转全流程
- 适合需要精细动作控制的家用机器人研发
多步骤灵巧操作是家庭场景中的基础技能，但在机器人领域仍研究不足。本文提出一种模块化方法，每个操作步骤采用基于有效模态输入的专用策略，而非依赖单一端到端模型。以机械手抓取并旋转盒子为例，结合神经科学启示，将任务分解为三个子技能：1）接近，2）抓取与提起，3）手中旋转，分别对应人脑中主导的感官模态。各子技能分别采用经典控制器、视觉-语言-动作模型和带力反馈的强化学习策略实现。在真实机器人上验证了该流程的可行性。核心贡献在于提出一种受神经科学启发的模态驱动多步灵巧操作方法。
原文摘要 · Abstract (English)
Multi-step dexterous manipulation is a fundamental skill in household scenarios, yet remains an underexplored area in robotics. This paper proposes a modular approach, where each step of the manipulation process is addressed with dedicated policies based on effective modality input, rather than relying on a single end-to-end model. To demonstrate this, a dexterous robotic hand performs a manipulation task involving picking up and rotating a box. Guided by insights from neuroscience, the task is decomposed into three sub-skills, 1)reaching, 2)grasping and lifting, and 3)in-hand rotation, based on the dominant sensory modalities employed in the human brain. Each sub-skill is addressed using distinct methods from a practical perspective: a classical controller, a Vision-Language-Action model, and a reinforcement learning policy with force feedback, respectively. We tested the pipeline on a real robot to demonstrate the feasibility of our approach. The key contribution of this study lies in presenting a neuroscience-inspired, modality-driven methodology for multi-step dexterous manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。