arXiv:2603.22263cs.RO2026-03

机器人用双手精准击鼓,实现长时间、高接触复杂操作。

DexDrummer: In-Hand, Contact-Rich, and Long-Horizon Dexterous Robot Drumming

  • 分层策略结合路径规划与强化学习修正,降低训练难度。
  • 仿真中多鼓演奏F1得分比固定抓握高1.87倍,真实世界演奏准确率1.0。
  • 适合研究复杂灵巧操作与人机协作的学者与工程师。

在手控制、高接触交互和长时程协调的灵巧操作仍是机器人领域的未解难题。以往研究多孤立处理其中某项挑战,未能整合为复杂任务。为此,我们以击鼓作为灵巧操作测试基准,因其天然融合了在手控制(稳定鼓槌)、高接触交互(反复敲击鼓面)和长时程协调(切换鼓具与持续节奏)。本文提出DexDrummer,一种基于模拟训练、支持现实迁移的分层对象中心双臂击鼓策略。通过轨迹规划结合残差强化学习,实现鼓间快速转移;灵巧操作策略通过显式建模手指-鼓槌与鼓槌-鼓面交互奖励,应对高接触动态。仿真结果表明,该策略可演奏两种风格:双鼓双手曲目与高难度技术练习。在双臂任务中,其F1得分相比固定抓握策略提升1.87倍(简单曲目)和1.22倍(复杂曲目)。真实场景中,系统成功完成多鼓演奏,训练曲及扩展版本的F1得分为1.0。

原文摘要 · Abstract (English)

Performing in-hand, contact-rich, and long-horizon dexterous manipulation remains an unsolved challenge in robotics. Prior hand dexterity works have considered each of these three challenges in isolation, yet do not combine these skills into a single, complex task. To further test the capabilities of dexterity, we propose drumming as a testbed for dexterous manipulation. Drumming naturally integrates all three challenges: it involves in-hand control for stabilizing and adjusting the drumstick with the fingers, contact-rich interaction through repeated striking of the drum surface, and long-horizon coordination when switching between drums and sustaining rhythmic play. We present DexDrummer, a hierarchical object-centric bimanual drumming policy trained in simulation with sim-to-real transfer. The framework reduces the exploration difficulty of pure reinforcement learning by combining trajectory planning with residual RL corrections for fast transitions between drums. A dexterous manipulation policy handles contact-rich dynamics, guided by rewards that explicitly model both finger-stick and stick-drum interactions. In simulation, we show our policy can play two styles of music: multi-drum, bimanual songs and challenging, technical exercises that require increased dexterity. Across simulated bimanual tasks, our dexterous, reactive policy outperforms a fixed grasp policy by 1.87x across easy songs and 1.22x across hard songs F1 scores. In real-world tasks, we show song performance across a multi-drum setup. DexDrummer is able to play our training song and its extended version with an F1 score of 1.0.

灵巧操作双臂控制仿真迁移音乐生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。