arXiv:2509.07635cs.SDcs.LG2025-09被引 3

用神经网络为合成器生成可感知的紧凑音色表示,让AI更好控制音乐合成。

Neural Proxies for Sound Synthesizers: Learning Perceptually Informed Preset Representations

  • 训练神经网络将合成器参数映射到预训练音频模型的声学空间。
  • 在三个主流合成器上测试,音色匹配任务表现良好,证明表示有效性。
  • 适合想用AI自动化编曲或研究音乐生成的研究者与创作者。

深度学习为自动合成器编程(ASP)提供了有前景的解决方案,旨在帮助音乐人和声音设计师配置声音合成器。然而,由于软件合成器可能不具备可微性,将其集成到训练流程中存在挑战。本文提出一种方法,通过训练神经网络将任意合成器的预设映射到由预训练模型生成的音频嵌入空间,构建神经代理,实现紧凑而有效的表示。该方法使音频嵌入损失可应用于黑箱合成器的神经基ASP系统中。我们在三个流行软件合成器的合成与手工预设上评估了不同预训练音频模型所生成表示的效果,并测试了前馈、循环及基于Transformer的多种神经网络架构在定义神经代理时的表现。在音色匹配下游任务中,所有合成器均获得令人鼓舞的结果,尽管资源开销影响表现程度,但仍为未来基于神经代理的合成器自动化研究铺平道路。

原文摘要 · Abstract (English)

Deep learning appears as an appealing solution for Automatic Synthesizer Programming (ASP), which aims to assist musicians and sound designers in programming sound synthesizers. However, integrating software synthesizers into training pipelines is challenging due to their potential non-differentiability. This work tackles this challenge by introducing a method to approximate arbitrary synthesizers. Specifically, we train a neural network to map synthesizer presets onto an audio embedding space derived from a pretrained model. This facilitates the definition of a neural proxy that produces compact yet effective representations, thereby enabling the integration of audio embedding loss into neural-based ASP systems for black-box synthesizers. We evaluate the representations derived by various pretrained audio models in the context of neural-based nASP and assess the effectiveness of several neural network architectures, including feedforward, recurrent, and transformer-based models, in defining neural proxies. We evaluate the proposed method using both synthetic and hand-crafted presets from three popular software synthesizers and assess its performance in a synthesizer sound matching downstream task. While the benefits of the learned representation are nuanced by resource requirements, encouraging results were obtained for all synthesizers, paving the way for future research into the application of synthesizer proxies for neural-based ASP systems.

音频生成神经代理合成器音色匹配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。