用音乐结构先验生成更协调的双人手钢琴动作，支持实时长序列生成。
PianoFlow: Music-Aware Streaming Piano Motion Generation with Bimanual Coordination
- 用MIDI符号信息训练，提升对乐理的理解力。
- 在PianoMotion10M数据集上推理速度提升9倍以上。
- 适合需要真实钢琴演奏动画的影视与游戏开发人员。
音频驱动的双人手钢琴动作生成需精准建模复杂音乐结构与动态跨手协作。现有方法多依赖仅含声学信息的表征，缺乏符号先验，交互机制僵化，且仅支持计算成本高的短序列生成。为此，我们提出PianoFlow，一种基于流匹配的精确双人手钢琴动作合成框架。训练时战略性利用MIDI作为优势模态，提炼结构化音乐先验以实现深层语义理解，同时保持仅音频输入的推理能力。此外，我们设计非对称角色门控交互模块，通过角色感知注意力与时间门控显式捕捉跨手动态协作。为实现任意长度序列的实时流式生成，我们提出自回归流延续方案，确保跨片段时间一致性。在PianoMotion10M数据集上的大量实验表明，PianoFlow在定量与定性指标上均优于现有方法，且推理速度提升超过9倍。
原文摘要 · Abstract (English)
Audio-driven bimanual piano motion generation requires precise modeling of complex musical structures and dynamic cross-hand coordination. However, existing methods often rely on acoustic-only representations lacking symbolic priors, employ inflexible interaction mechanisms, and are limited to computationally expensive short-sequence generation. To address these limitations, we propose PianoFlow, a flow-matching framework for precise and coordinated bimanual piano motion synthesis. Our approach strategically leverages MIDI as a privileged modality during training, distilling these structured musical priors to achieve deep semantic understanding while maintaining audio-only inference. Furthermore, we introduce an asymmetric role-gated interaction module to explicitly capture dynamic cross-hand coordination through role-aware attention and temporal gating. To enable real-time streaming generation for arbitrarily long sequences, we design an autoregressive flow continuation scheme that ensures seamless cross-chunk temporal coherence. Extensive experiments on the PianoMotion10M dataset demonstrate that PianoFlow achieves superior quantitative and qualitative performance, while accelerating inference by over 9\times compared to previous methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。