arXiv:2604.11052cs.SD2026-04中稿 · ACMMM 2026

用扩散模型生成伴奏,兼顾音色真实、结构连贯与动态编排。

LaDA-Band: Language Diffusion Models for Vocal-to-Accompaniment Generation

论文配图:LaDA-Band: Language Diffusion Models for Vocal-to-Accompaniment Generation
图 1 · 摘自论文原文
  • 采用离散掩码扩散框架,双向建模全曲上下文。
  • 在真实数据上显著提升音色清晰度与整体一致性。
  • 无需参考音频也能生成高质量伴奏,适合音乐创作应用。

Vocal-to-accompaniment(V2A)生成旨在将原始人声转化为完整编排的伴奏,需同时解决三大挑战:保持声学真实性、与人声轨道全局一致、实现整首歌的动态编排。现有开源方法常在三者间妥协:连续潜变量模型虽能捕捉长序列,却难以保留精细声学细节;离散自回归模型虽保有局部保真度,但存在单向生成和误差累积问题。本文提出LaDA-Band,首次将离散掩码扩散模型引入V2A任务。该方法将生成建模为离散掩码扩散过程,结合离散音频编码器的表示优势与全序列双向上下文建模能力,显著提升长时结构一致性与时间同步性,同时保留清晰声学细节。在此基础上,引入双轨前缀条件架构、弱锚定区域替换标记检测辅助目标,以及两阶段渐进式课程训练策略,以支持整首歌规模的生成。在学术与真实世界基准上的大量实验表明,LaDA-Band在声学真实性、全局一致性与动态编排方面均优于现有基线,且无需辅助参考音频仍具强表现力。代码与音频样本见https://github.com/Duoluoluos/TME-LaDA-Band。

原文摘要 · Abstract (English)

Vocal-to-accompaniment (V2A) generation, which aims to transform a raw vocal recording into a fully arranged accompaniment, inherently requires jointly addressing an accompaniment trilemma: preserving acoustic authenticity, maintaining global coherence with the vocal track, and producing dynamic orchestration across a full song. Existing open-source approaches typically make compromises among these goals. Continuous-latent generation models can capture long musical spans but often struggle to preserve fine-grained acoustic detail. In contrast, discrete autoregressive models retain local fidelity but suffer from unidirectional generation and error accumulation in extended contexts. We present LaDA-Band, an end-to-end framework that introduces Discrete Masked Diffusion to the V2A task. Our approach formulates V2A generation as Discrete Masked Diffusion, i.e., a global, non-autoregressive denoising formulation that combines the representational advantages of discrete audio codec tokens with full-sequence bidirectional context modeling. This design improves long-range structural consistency and temporal synchronization while preserving crisp acoustic details. Built on this formulation, LaDA-Band further introduces a dual-track prefix-conditioning architecture, an auxiliary replaced-token detection objective for weakly anchored accompaniment regions, and a two-stage progressive curriculum to scale Discrete Masked Diffusion to full-song vocal-to-accompaniment generation. Extensive experiments on both academic and real-world benchmarks show that LaDA-Band consistently improves acoustic authenticity, global coherence, and dynamic orchestration over existing baselines, while maintaining strong performance even without auxiliary reference audio. Codes and audio samples are available at https://github.com/Duoluoluos/TME-LaDA-Band .

音乐生成扩散模型音频处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。