arXiv:2604.23886cs.GRcs.AI2026-04被引 1

用肌肉驱动模型让机械手精准弹奏新曲目,突破数据限制。

MUSIC: Learning Muscle-Driven Dexterous Hand Control

论文配图:MUSIC: Learning Muscle-Driven Dexterous Hand Control
图 1 · 摘自论文原文
  • 分层控制:低频潜空间协调,高频肌肉激活追踪动作。
  • 可生成新乐曲演奏,实现物理仿真下最先进钢琴表现。
  • 模型更贴近人体肌电规律,适合仿生机器人研究者。

我们提出一种数据驱动的物理仿真、肌肉驱动的灵巧手控制方法,使肌骨骼手能够演奏参考数据集之外的新乐曲。该方法采用分层架构,低层通过强化学习训练通用单手策略,生成动态肌肉-肌腱激活以跟踪大规模参考运动数据集中的轨迹;随后将这些跟踪策略蒸馏为变分自编码器(VAE)模型,获得平滑且结构化的潜空间,抽象出底层肌肉动态。高层则训练针对具体乐曲的策略,在此潜空间中操作,基于乐谱提取的音符事件协调双手动作,合成超出参考数据范围的演奏。此外,我们还构建了一个增强型肌骨骼手模型,支持手指精细控制,提升低层运动追踪精度与高层动作多样性。在涵盖多种音乐风格与技术难度的钢琴曲目上评估表明,该方法能生成协调的双手动作和精确的按键表现,达到物理仿真灵巧控制领域的最先进水平。同时,新模型在生物力学稳定性和运动追踪精度上优于现有模型,并生成与人类肌电图(EMG)记录一致的生理学合理激活模式。

原文摘要 · Abstract (English)

We present a data-driven approach for physics-based, muscle-driven dexterous control that enables musculoskeletal hands to perform precise piano playing for novel pieces of music outside the reference dataset. Our approach combines high-frequency muscle-level control with low-frequency latent-space coordination in a hierarchical architecture. At the low level, general single-hand policies are trained via reinforcement learning to generate dynamic muscle-tendon activations while tracking trajectories from a large reference motion dataset. The resulting tracking policies are then distilled into variational autoencoder (VAE) models, yielding smooth and structured latent spaces that abstract away low-level muscle dynamics. For the high level, we train piece-specific policies to operate in this latent space, coordinating bimanual motions based on specific goals, denoted by note events extracted from given musical scores, to synthesize performances beyond the reference data. In addition, we present an enhanced musculoskeletal hand model that supports fine control of fingers for accurate low-level motion tracking and diverse high-level motion synthesis. We evaluate the control pipeline of our approach on a diverse piano repertoire spanning multiple musical styles and technical demands. Results demonstrate that our approach can synthesize coordinated bimanual motions with accurate key presses, and achieve the state-of-the-art performance of piano playing in physics-based dexterous control. We also show that our musculoskeletal hand model demonstrates superior biomechanical stability and tracking precision compared to the existing model, and validate that our musculoskeletal hand model and muscle-driven controller can generate physiologically plausible activation patterns that align with human electromyography (EMG) recordings.

肌肉控制灵巧手钢琴生成生物力学

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。