让音频包络插值更符合人耳感知,生成自然过渡音效。
Learning Perceptually Relevant Temporal Envelope Morphing
- 通过听觉实验提炼出感知驱动的包络插值规则
- 构建大规模数据集并训练模型实现感知中间态合成
- 适用于音乐创作与心理声学研究,提升音效融合自然度
时间包络插值是生成音频系统中的新兴问题,但现有方法缺乏感知基础。当输入音频具有不同时间结构时,传统方法常叠加两者特征,导致听感不自然。本文提出新流程:首先通过人类听觉实验提取感知驱动的插值原则,再构建大规模数据集编码这些原则,最后训练机器学习模型生成感知上居中的中间形态。具体包括:(1) 从听觉实验中得出的包络插值感知原则;(2) 基于监督学习的模型框架;(3) 用于压缩包络结构的自编码器;(4) 使用合成与真实数据的评估基准。结果表明,该方法在生成时间中间态包络方面优于现有技术。代码、模型及检查点均开源。
原文摘要 · Abstract (English)
Temporal envelope morphing, the process of interpolating between the amplitude dynamics of two audio signals, is an emerging problem in generative audio systems that lacks sufficient perceptual grounding. Morphing of temporal envelopes in a perceptually intuitive manner should enable new methods for sound blending in creative media and for probing perceptual organization in psychoacoustics. However, existing audio morphing techniques often fail to produce intermediate temporal envelopes when input sounds have distinct temporal structures; many morphers effectively overlay both temporal structures, leading to perceptually unnatural results. In this paper, we introduce a novel workflow for learning envelope morphing with perceptual guidance: we first derive perceptually grounded morphing principles through human listening studies, then synthesize large-scale datasets encoding these principles, and finally train machine learning models to create perceptually intermediate morphs. Specifically, we present: (1) perceptual principles that guide envelope morphing, derived from our listening studies, (2) a supervised framework to learn these principles, (3) an autoencoder that learns to compress temporal envelope structures into latent representations, and (4) benchmarks for evaluating audio envelope morphs, using both synthetic and naturalistic data, and show that our approach outperforms existing methods in producing temporally intermediate morphs. All code, models, and checkpoints are available at https://github.com/TemporalMorphing/EnvelopeMorphing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。