实时生成伴奏，让演奏者即兴伴奏不卡顿。
LiveBand: Live Accompaniment Generation in the Audio Domain
- 用因果Transformer在音频潜空间生成伴奏，仅依赖当前及过去输入。
- 在多乐器数据集上音质、节奏对齐和混音贴合度均优于以往方法。
- 支持消费级硬件实时流式生成，无未来信息窥探，适合现场演出场景。
我们提出LiveBand，一个实时系统，可为实时音频输入生成高保真音乐伴奏，严格遵守因果约束。该方法在预训练的因果音频自编码器的连续潜空间中训练因果Transformer生成器，使用来自判别器的对抗性序列级监督。每个时间步，生成器仅接收因果可用的混合上下文和高斯噪声，预测伴奏潜变量，无法访问未来的混合帧或真实目标潜变量。训练采用因果掩码进行单次并行前向传播，而流式推理则通过滚动注意力状态自回归进行。模型的训练与推理计算设计一致，消除了教师强制及其带来的暴露偏差。在多乐器音乐伴奏基准上，LiveBand在客观音频质量、节拍对齐和混音贴合度方面优于先前方法，同时可在消费级硬件上实现无前瞻的实时流式生成。
原文摘要 · Abstract (English)
We present LiveBand, a real-time system that generates high-fidelity music accompaniments to live audio input, respecting strict causal constraints. Our method trains a causal transformer generator in the continuous latent space of a pre-trained causal audio autoencoder, using adversarial sequence-level supervision from a discriminator. At each timestep, the generator receives only the causally available mix context and Gaussian noise, and predicts accompaniment latents without access to future mix frames or ground-truth target latents. Training is performed in a single parallel forward pass under causal masking, while streaming inference proceeds autoregressively with a rolling attention state. The model's training and inference computations are matched by design, eliminating teacher forcing and the associated exposure bias. On a multi-instrument music accompaniment benchmark, LiveBand improves over prior work on objective measures of audio quality, beat alignment, and mix adherence, while enabling real-time streaming generation without lookahead into the future on consumer hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。