用轻量模型实现环境音高保真生成,支持语义控制与平滑切换。
SCAPES: Semantically Conditioned Autoregressive Prior for Environmental Sounds

- 基于连续潜空间的自回归先验,避免离散分词限制。
- 3600万参数模型在单张消费级显卡上训练,收敛时间约源音频时长两倍。
- 支持语义插值,适合研究与创意音效设计,开源可试用。
随着生成音频模型日益复杂,合成日常环境声音的计算与生态成本已变得难以承受,通常需要工业级资源和大规模数据集。本文提出SCAPES:一种面向环境声音的语义条件自回归先验。该模型轻量高效,通过神经音频编码器的连续潜空间进行操作,避免了离散分词带来的刚性结构约束。我们提出一种重叠分段策略,利用连续归一化流(CNF)结合流匹配(Flow Matching)建模潜变量轨迹演化。实验表明,一个3600万参数的SCAPES实例可在有限、未标注数据集上仅用单张消费级显卡完成训练,约需两倍于源音频时长的训练时间即可收敛,生成具有高保真度、长期稳定性和语义一致性的音频输出。此外,模型展现出平滑的语义插值能力,为开放研究与创意音效设计提供灵活且易用的工具。代码、预训练权重、音频示例及交互式演示已公开于项目页面 https://cordutie.github.io/projects/scapes.html。
原文摘要 · Abstract (English)
As generative audio models grow in complexity, the computational and ecological costs of synthesizing everyday sounds have become increasingly prohibitive, often requiring industrial-scale resources and massive datasets. In this paper, we present SCAPES: a Semantically Conditioned Autoregressive Prior for Environmental Sounds. SCAPES is a lightweight, resource-efficient generative model designed to synthesize high-fidelity environmental textures through high-level semantic control. By operating on the continuous latent manifold of a neural audio codec, our approach bypasses the rigid structural constraints inherent to discrete tokenization. We propose a segmentation strategy that decomposes audio into overlapping segments, enabling a Continuous Normalizing Flow (CNF) to model the evolution of latent trajectories using Flow Matching. Our experiments demonstrate that a 36-million parameter instance of SCAPES can be trained on limited, uncurated datasets using a single consumer-grade GPU. Notably, convergence is achieved after training for approximately twice the source audio duration, yielding high-fidelity outputs with robust long-term stability and semantic consistency. Furthermore, we showcase the model's capacity for smooth semantic interpolation, providing a flexible and accessible tool for open research and creative sound design. Code, pretrained weights, audio examples, and an interactive demo are publicly available on our project page https://cordutie.github.io/projects/scapes.html
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。