用高斯先验优化音效风格迁移，让结果更真实可信。
Improving Inference-Time Optimisation for Vocal Effects Style Transfer with a Gaussian Prior
- 引入基于数据集的高斯先验，约束参数空间搜索范围。
- 在MedleyDB上使参数均方误差降低33%，更贴近参考音效。
- 适合音效迁移、音频处理等需要真实参数生成的场景。
基于推理时优化的风格迁移（ST-ITO）是一种将参考音频的音效特征迁移到目标音频的方法，通过优化音效参数以最小化处理后音频与参考音频在风格嵌入空间中的距离。然而，该方法对所有参数配置一视同仁，仅依赖嵌入空间，易导致不现实或有偏的参数配置。本文提出从DiffVox声乐预设数据集导出的高斯先验，作用于参数空间，使优化过程等价于最大后验估计。在MedleyDB上的评估显示，相比盲音效估计器、最近邻方法及未校准的ST-ITO，本方法在多项指标上均有显著提升：参数均方误差最高降低33%，且更贴近参考风格。16名参与者参与的主观评估也证实了该方法在数据有限场景下的优越性。结果表明，推理时引入先验知识能有效提升音效迁移效果，推动更高效、真实的音频处理系统发展。
原文摘要 · Abstract (English)
Style Transfer with Inference-Time Optimisation (ST-ITO) is a recent approach for transferring the applied effects of a reference audio to an audio track. It optimises the effect parameters to minimise the distance between the style embeddings of the processed audio and the reference. However, this method treats all possible configurations equally and relies solely on the embedding space, which can result in unrealistic configurations or biased outcomes. We address this pitfall by introducing a Gaussian prior derived from the DiffVox vocal preset dataset over the parameter space. The resulting optimisation is equivalent to maximum-a-posteriori estimation. Evaluations on vocal effects transfer on the MedleyDB dataset show significant improvements across metrics compared to baselines, including a blind audio effects estimator, nearest-neighbour approaches, and uncalibrated ST-ITO. The proposed calibration reduces the parameter mean squared error by up to 33% and more closely matches the reference style. Subjective evaluations with 16 participants confirm the superiority of our method in limited data regimes. This work demonstrates how incorporating prior knowledge at inference time enhances audio effects transfer, paving the way for more effective and realistic audio processing systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。