通过自回归预测音轨效果链,学习音频混音风格表示。
StemFX: Learning Mixing Style Representations via Autoregressive FX Chain Prediction on Source-Separated Stems
- 用Transformer自回归生成可变长度效果链,结合频带分割CNN编码器捕捉音轨频谱特征。
- 在10万首歌曲的伪音轨上训练,支持任意数量和类型的音频效果组合。
- 效果迁移速度超优化方法4000倍,音质更佳且听众偏好最高。
音频混音风格包含混音工程师在电平平衡、空间分布及每个音轨上音频效果(FX)的选择、排序与参数设置等方面的艺术与技术决策。效果链是决定混音风格的关键因素,但现有建模方法仍受限:部分方法基于立体声混合物,未显式建模每条音轨的效果链;另一些固定每轨效果数量或类型;多数需要可微分效果实现或稀缺的多轨数据集。本文提出StemFX框架,通过自回归方式在源分离音轨上学习混音风格表示。采用Transformer解码器自回归预测分词后的效果链,同时使用带FiLM条件的频带分割多频带CNN编码器捕捉每条音轨的频谱结构。为支持大规模成对训练,我们利用源分离从约10.5万首歌曲中提取伪音轨,并通过MultiAFx工具包(整合7个Python库中的85种音频效果)进行增强。在混音风格检索任务中,StemFX在所有测试链长下均优于所有基线模型;在成对混音风格迁移任务中,其达到最佳频谱保真度与最高听觉偏好,且速度超过迭代优化4000倍以上。
原文摘要 · Abstract (English)
Audio mixing style encompasses the artistic and technical decisions a mix engineer makes, including level balancing, spatialization, and the choice, ordering, and parameterization of audio effects (FX) on each stem. FX chains are a key determinant of this style, yet existing approaches to modeling them remain limited. Some operate on stereo mixtures without explicit per-stem FX chain modeling, others fix the number or type of effects per track, and many require differentiable effect implementations or scarce multitrack datasets. We present StemFX, a framework that learns mixing style representations by autoregressively predicting variable-length FX chains on source-separated stems. A Transformer decoder predicts tokenized FX chains autoregressively, while a band-split multi-band CNN encoder with FiLM conditioning captures per-stem spectral structure. To enable large-scale paired training, we extract pseudo-stems from about 105K songs via source separation and augment them using MultiAFx, a toolkit unifying 85 audio effects from 7 Python libraries. Evaluated on mixing style retrieval, StemFX outperforms all baseline models across all tested chain lengths. On paired mixing style transfer, StemFX achieves the best spectral fidelity and the highest listener preference, over 4000 times faster than iterative optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。