arXiv:2602.19816cs.SDcs.AI2026-02

提出新方法实现音乐模型整首曲子训练,提升长程依赖建模能力。

Whole-Piece Training for Symbolic Music Language Models via Full-Horizon Compressed Recurrence

  • 通过压缩循环记忆的键值表示,保持完整时间跨度
  • 在MAESTRO数据集上验证模型能利用远超局部窗口的上下文
  • 适合关注音乐全局结构建模的研究者与创作者

为提升计算效率,现代语言模型通常基于独立采样的固定长度序列进行训练。符号化音乐语言模型也沿用此范式,尽管音乐结构天然随完整作品展开而非孤立片段。将作品切分为独立训练样本会阻碍对整部作品的连续条件建模。本文提出全时域压缩循环(FHCR)框架,实现符号化音乐语言模型的整首曲子训练。FHCR在保留循环记忆完整时间跨度的同时,压缩其键值(KV)表示维度,使在有限GPU内存下持续训练成为可能。为直接评估长程依赖能力,我们引入评估期诊断工具KV-重置上下文利用率(KRCU)。在MAESTRO符号钢琴数据集上,KRCU显示全时域模型可利用远超局部段窗的上下文,而缩减循环记忆时长则显著削弱这种可测量的长程依赖。FHCR在大幅降低循环记忆开销的同时,仍保持长程上下文利用率。结果表明,保持循环历史的时间范围对高效整首建模至关重要,而内存开销可通过压缩KV表示来降低。项目演示与生成样例见https://wholemusic.github.io。

原文摘要 · Abstract (English)

For computational efficiency, modern language models are typically trained on independently sampled fixed-length sequences. Symbolic music language models largely inherit this paradigm, despite musical structure naturally unfolding over complete compositions rather than isolated excerpts. Fragmenting compositions into independent training instances therefore prevents continuous conditioning over the complete work. We present a practical framework for whole-piece training of symbolic music language models via Full-Horizon Compressed Recurrence (FHCR). FHCR preserves the full temporal horizon of recurrent memory while reducing the dimensionality of its key-value (KV) representation, making continuous whole-piece training practical under limited GPU memory. To directly assess functional long-range dependence, we introduce KV-Reset Context Utilization (KRCU), an evaluation-time diagnostic. On the MAESTRO symbolic piano dataset, KRCU shows that full-horizon models utilize context far beyond the local segment window, whereas reducing the temporal extent of recurrent memory substantially weakens this measurable long-range dependence. FHCR preserves long-range context utilization while substantially reducing recurrent memory cost. These findings show that preserving the temporal extent of recurrent history is important for efficient whole-piece modeling, and that memory cost can instead be reduced through KV representation compression. The project demos and generated music samples are available at https://wholemusic.github.io.

音乐生成长程依赖序列建模模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。