用100亿标记的无标签乐谱数据预训练，让钢琴演奏更自然。
Pianist Transformer: Towards Expressive Piano Performance Rendering via Scalable Self-Supervised Pre-Training
- 基于统一MIDI表示，用100亿标记的无标签乐谱做自监督预训练。
- 采用笔记级压缩的非对称Transformer，显著提升长音乐序列建模效率。
- 支持可编辑工作流，适合音乐制作人直接集成到实际创作中。
现有表达性音乐演奏生成方法依赖小规模标注数据的监督学习，限制了数据量与模型规模的扩展，尽管视觉和语言领域已拥有海量未标注数据。为填补这一空白，本文提出Pianist Transformer，包含三项关键贡献：1）首次在表达性钢琴演奏生成中引入大规模自监督学习，基于统一的音乐设备数字接口（MIDI）表示，实现对100亿标记无标签MIDI数据的预训练；2）设计一种高效的非对称Transformer架构，通过笔记级压缩大幅降低训练开销、内存占用与推理延迟，适用于长上下文音乐建模；3）构建当前最优的演奏渲染模型，具备可编辑工作流，取得优异客观与主观评估结果，并可无缝融入真实音乐制作流程。整体上，该框架为音乐领域实现类人演奏合成提供了可扩展路径。代码、音频样例及模型权重见项目主页：https://yhj137.github.io/pianist-transformer-demo/。
原文摘要 · Abstract (English)
Existing methods for expressive music performance rendering, a conditional generation task that aims to generate a human-like performance from a symbolic score, rely on supervised learning over small labeled datasets, which limits scaling of both data volume and model size, despite the availability of vast unlabeled music, as in vision and language. To address this gap, we introduce Pianist Transformer, with three key contributions: 1) introducing large-scale self-supervised learning into expressive piano performance rendering through a unified Musical Instrument Digital Interface (MIDI) representation, enabling pre-training on 10B tokens of unlabeled MIDI data; 2) an efficient asymmetric Transformer with note-level compression, substantially improving training efficiency, memory usage, and inference speed for long-context music modeling; 3) a state-of-the-art rendering model with an editable workflow, achieving strong objective and subjective results and enabling integration into real-world music production workflows. Overall, Pianist Transformer outlines a scalable path toward human-like performance synthesis in the music domain. Code, audio samples, and model checkpoints are available on our project page: https://yhj137.github.io/pianist-transformer-demo/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。