让音乐模型实时配合人声旋律，实现在线即兴伴奏。
Adaptive Accompaniment with ReaLchords
- 用强化学习优化在线生成模型，实现与旋律同步伴奏。
- 新奖励模型同时评估和声与时间协调性，提升伴奏贴合度。
- 适合音乐创作、实时协作场景，支持人机共同即兴演奏。
即兴演奏需要乐手间的协调、预判与共创。现有音乐生成模型虽具表现力，但无法实时生成（即与他人同步）。 我们提出 ReaLchords，一种用于即兴和弦伴奏的在线生成模型。以最大似然预训练的在线模型为基础，采用强化学习进行微调，目标函数结合了新颖的奖励模型（评估旋律与和弦在和声与时间上的连贯性）以及一项新型的蒸馏项（从可预见未来旋律的教师模型中学习）。通过定量实验与听感测试，结果表明该模型能良好适应陌生输入并生成契合的伴奏。ReaLchords 为现场合奏及多模态协同创作开辟了可能。
原文摘要 · Abstract (English)
Jamming requires coordination, anticipation, and collaborative creativity between musicians. Current generative models of music produce expressive output but are not able to generate in an \emph{online} manner, meaning simultaneously with other musicians (human or otherwise). We propose ReaLchords, an online generative model for improvising chord accompaniment to user melody. We start with an online model pretrained by maximum likelihood, and use reinforcement learning to finetune the model for online use. The finetuning objective leverages both a novel reward model that provides feedback on both harmonic and temporal coherency between melody and chord, and a divergence term that implements a novel type of distillation from a teacher model that can see the future melody. Through quantitative experiments and listening tests, we demonstrate that the resulting model adapts well to unfamiliar input and produce fitting accompaniment. ReaLchords opens the door to live jamming, as well as simultaneous co-creation in other modalities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。