让吉他音频从干净音色变出指定效果,保持原曲内容不变
Clean2FX: Label-conditioned modeling for clean-to-effect guitar audio transformations
- 用标签控制音色变换,输入干净吉他声+目标效果标签
- U-Net模型表现最好,失真效果改善最明显
- 适合音乐制作人和音频工程师做真实演奏的实时效果处理
我们提出Clean2FX,研究并演示了基于标签的电吉他音频从干净到效果的转换。给定一段干净的吉他音频和目标效果标签,任务是合成对应的效果信号,同时保留音乐内容。训练与评估数据对来自EGFxSet真实单音录制,通过组合匹配的干净/效果和弦、旋律及混合时间线构建,实现不同效果间的可控对比。在统一的频谱变换框架下评估四种神经方法:两种变分自编码器和两种在线性或对数幅值表示上运行的U-Net模型。性能以线性幅值频谱的MSE和弗雷切特音频距离(FAD)衡量。U-Net模型优于变分自编码器。各效果结果表明,失真效果最易提升,而延迟和混响虽频谱误差显著降低,但FAD增益较弱。条件敏感性诊断显示最佳模型能响应目标标签,而非退化为单一变换。演示网站展示了两个模型在训练和验证数据外的真实演奏上的应用,提供音频与频谱示例,展示实际转换效果。
原文摘要 · Abstract (English)
We present Clean2FX, a study and demo of label-conditioned clean-to-effect transformation for electric guitar audio. Given a clean guitar input and a target effect label, the task is to synthesize the corresponding effected signal while preserving the musical content. Training and evaluation pairs are constructed from EGFxSet real, single tone recordings by assembling matched clean/effected chords, melodies, and mixed timelines. This allows for controlled comparison across effects. We evaluate four neural approaches under a common spectrogram-based transformation setting: two variational autoencoders and two U-Net models that differ in whether they operate on linear or log-magnitude representations. Performance is measured using linear-magnitude spectrogram MSE and Fréchet Audio Distance. The U-Net models outperform the variational autoencoder variants. Per-effect results show that distortion effects are most readily improved, whereas delay and reverb effects exhibit weaker FAD gains despite substantial spectral-error reductions. A conditioning-sensitivity diagnostic provides evidence that the best model responds to target labels rather than collapsing to a single transformation. Our demo website compares two models applied on real-world guitar performances outside training and validation data, providing audio and spectrogram examples of the practical clean-to-effect behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。