让音乐中特定声部变音色，背景其他声部不变，零样本直接实现。
Polyphonia: Zero-Shot Timbre Transfer in Polyphonic Music with Acoustic-Informed Attention Calibration
- 用声学先验校准注意力，精准定位复杂混音中的目标声部。
- 在1170个任务上，目标对齐率比基线提升15.5%。
- 适合需要精细控制音乐元素的创作者和研究者。
基于扩散模型的文本到音乐生成取得了进展，为零样本音乐编辑开辟了新路径。然而，现有方法无法实现针对特定声部的音色迁移，即在严格保留背景伴奏的前提下改变特定声部音色。这一限制严重制约了实际应用，因为真实制作中需对密集混音中的成分进行精确操控。我们的关键发现是：虽然原始交叉注意力能捕捉声部的语义特征，但缺乏频谱分辨率，难以在密集混合中精确定位目标，导致边界泄漏。为解决此矛盾，我们提出Polyphonia，一种基于声学引导注意力校准的零样本编辑框架。该方法不依赖单一的语义注意力，而是利用概率声学先验建立粗略边界，从而确保非目标声部的语义合成精度。为评估，我们构建了PolyEvalPrompts，一个包含1170个多声部音乐音色迁移任务的标准提示集。实验表明，Polyphonia在目标对齐方面相较基线提升15.5%，同时保持了良好的音乐保真度与非目标完整性。
原文摘要 · Abstract (English)
The advancement of diffusion-based text-to-music generation has opened new avenues for zero-shot music editing. However, existing methods fail to achieve stem-specific timbre transfer, which requires altering specific stems while strictly preserving the background accompaniment. This limitation severely hinders practical application, since real-world production necessitates precise manipulation of components within dense mixtures. Our key finding is that, while vanilla cross-attention captures semantic features of stems, it lacks the spectral resolution to strictly localize targets in dense mixtures, leading to boundary leakage. To resolve this dilemma, we propose Polyphonia, a zero-shot editing framework with Acoustic-Informed Attention Calibration. Rather than relying solely on diffuse semantic attention, Polyphonia leverages a probabilistic acoustic prior to establish coarse boundaries, enabling non-target stems preserved precise semantic synthesis. For evaluation, we propose PolyEvalPrompts, a standardized prompt set with 1,170 timbre transfer tasks in polyphonic music. Specifically, Polyphonia achieves an increase of 15.5% in target alignment compared to baselines, while maintaining competitive music fidelity and non-target integrity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。