用图像扩散模型实现无需训练的音乐风格迁移，保留原曲结构同时提升音质。
Repurposing Image Diffusion Models for Training-Free Music Style Transfer on Mel-spectrograms
- 将音频转为时频图，用图像扩散模型注入风格信息而不改原结构
- 人评结果显示内容保留率高34.1%，感知质量提升25.7%
- 无需训练，适合快速个性化音乐创作，尤其适合非专业用户
音乐风格迁移旨在融合源音乐结构与参考风格，实现个性化音乐生成。现有零样本方法常因依赖粗略文本描述或需昂贵的任务定制训练，难以捕捉细微音频特征。本文提出 Stylus，一种无需训练的框架，复用预训练图像扩散模型完成梅尔频谱域的音乐风格迁移。通过将音频视为结构化时频图像，Stylus 在保持源结构查询的同时，通过注入风格键值对操控自注意力机制。为保证高保真度，引入相位保持重建策略以缓解频谱反演伪影，并采用类无分类器引导控制实现可调风格强度。大规模评估包含2,925次人工评分，结果表明 Stylus 显著优于现有基线，内容保留率提升34.1%，感知质量改善25.7%。本工作验证了通用图像先验可有效用于结构化梅尔频谱的免训练转换。代码与材料见 https://github.com/Sooyyoungg/Stylus.git。
原文摘要 · Abstract (English)
Music style transfer blends source structure with reference style to enable personalized music creation. However, existing zero-shot methods often struggle to capture fine-grained audio nuances, relying on coarse text descriptions or requiring expensive task-specific training. We propose Stylus, a training-free framework that repurposes pretrained image diffusion models for music style transfer in the Mel-spectrogram domain. By treating audio as structured time-frequency images, Stylus manipulates self-attention by injecting style keys and values while preserving source structural queries. To ensure high fidelity, we introduce a phase-preserving reconstruction strategy to mitigate spectrogram inversion artifacts, alongside a classifier-free-guidance-inspired control for adjustable stylization. Extensive evaluations including 2,925 human ratings demonstrate that Stylus outperforms state-of-the-art baselines, achieving 34.1% higher content preservation and 25.7% better perceptual quality. Our work validates that generic image priors can be effectively leveraged for the training-free transformation of structured Mel-spectrograms. Code and materials are available at https://github.com/Sooyyoungg/Stylus.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。