arXiv:2507.01582cs.SDcs.AI2025-07中稿 · IEEE SMC 2025被引 1

用双分支模型生成有表现力的古典钢琴演奏,兼具作曲与演奏双重创造力。

Exploring Classical Piano Performance Generation with Expressive Music Variational AutoEncoder

  • 设计双分支VAE模型,分别模拟作曲家与演奏家角色。
  • 在多个评估中优于现有模型,表现力与音乐质量更佳。
  • 适合音乐生成、创造性计算研究者参考。

古典音乐的创造性不仅源于作曲家创作乐谱,也来自演奏者对静态记谱的富有表现力的诠释。本文提出从零生成古典钢琴演奏的挑战,旨在模拟作曲与演奏的双重创造过程。我们引入表达性复合词(ECP)表示法,有效捕捉演奏的节拍结构与表现细节。在此基础上,提出表达性音乐变分自编码器(XMVAE),包含两个分支:基于向量量化变分自编码器(VQ-VAE)的分支生成乐谱内容(作曲家角色),以及普通VAE分支生成表现细节(演奏家角色)。两分支采用相似的序列到序列架构,利用多尺度编码器捕捉节拍级上下文信息,并通过正交Transformer解码器高效解码复合标记。客观与主观评估均表明,相比当前最优模型,XMVAE生成的演奏具有更优的音乐质量。此外,对作曲家分支在额外乐谱数据集上进行预训练,带来显著性能提升。

原文摘要 · Abstract (English)

The creativity of classical music arises not only from composers who craft the musical sheets but also from performers who interpret the static notations with expressive nuances. This paper addresses the challenge of generating classical piano performances from scratch, aiming to emulate the dual roles of composer and pianist in the creative process. We introduce the Expressive Compound Word (ECP) representation, which effectively captures both the metrical structure and expressive nuances of classical performances. Building on this, we propose the Expressive Music Variational AutoEncoder (XMVAE), a model featuring two branches: a Vector Quantized Variational AutoEncoder (VQ-VAE) branch that generates score-related content, representing the Composer, and a vanilla VAE branch that produces expressive details, fulfilling the role of Pianist. These branches are jointly trained with similar Seq2Seq architectures, leveraging a multiscale encoder to capture beat-level contextual information and an orthogonal Transformer decoder for efficient compound tokens decoding. Both objective and subjective evaluations demonstrate that XMVAE generates classical performances with superior musical quality compared to state-of-the-art models. Furthermore, pretraining the Composer branch on extra musical score datasets contribute to a significant performance gain.

音乐生成变分自编码器表现力钢琴演奏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。