用双扩散桥实现无监督音色迁移，保留旋律更佳。
Latent Diffusion Bridges for Unsupervised Musical Audio Timbre Transfer
- 通过双扩散模型映射音色到高斯先验并重建目标音色。
- FAD更低、音高距离更小，音色迁移与旋律保持更优。
- 调节噪声水平可控制音色改变程度和旋律保留强度。
音乐音色迁移是一项挑战性任务，需在保持旋律结构的同时修改音频的音色特征。本文提出一种基于双扩散桥的新方法，使用CocoChorales数据集（包含未配对的单乐器独奏音频）进行训练，每个扩散模型针对特定乐器，采用高斯先验。推理时，一个模型作为源模型将输入音频映射至其对应的高斯先验，另一个作为目标模型从该先验重建目标音频，从而实现音色迁移。与现有无监督音色迁移模型如VAEGAN和高斯流桥（GFB）相比，实验表明本方法在弗雷切特音频距离（FAD）和旋律保持性（以音高距离DPD衡量）上均表现更优。此外，发现高斯先验的噪声水平σ可调节，用于控制旋律保留程度与音色迁移量。
原文摘要 · Abstract (English)
Music timbre transfer is a challenging task that involves modifying the timbral characteristics of an audio signal while preserving its melodic structure. In this paper, we propose a novel method based on dual diffusion bridges, trained using the CocoChorales Dataset, which consists of unpaired monophonic single-instrument audio data. Each diffusion model is trained on a specific instrument with a Gaussian prior. During inference, a model is designated as the source model to map the input audio to its corresponding Gaussian prior, and another model is designated as the target model to reconstruct the target audio from this Gaussian prior, thereby facilitating timbre transfer. We compare our approach against existing unsupervised timbre transfer models such as VAEGAN and Gaussian Flow Bridges (GFB). Experimental results demonstrate that our method achieves both better Fréchet Audio Distance (FAD) and melody preservation, as reflected by lower pitch distances (DPD) compared to VAEGAN and GFB. Additionally, we discover that the noise level from the Gaussian prior, $σ$, can be adjusted to control the degree of melody preservation and amount of timbre transferred.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。