arXiv:2507.07867cs.SDcs.LG2025-07中稿 · IEEE MLSP 2025被引 6

通过重构瓶颈,让音频自编码器的潜在表示更符合下游应用需求。

Re-Bottleneck: Latent Re-Structuring for Neural Audio Autoencoders

  • 在预训练模型上添加内层瓶颈,仅用潜在空间损失优化结构。
  • 实现通道有序、语义对齐与变换等不变性,重建质量不受损。
  • 无需重训练,适配语音合成、音频生成等多样化任务。

神经音频编解码器和自编码器已成为音频压缩、传输、特征提取及潜在空间生成的通用模型。然而,多数模型仅追求重建保真度,忽视了下游应用所需的特定潜在结构。本文提出一种简单、后处理式的框架,通过修改预训练自编码器的瓶颈层来解决此问题。该方法引入“Re-Bottleneck”——一个仅通过潜在空间损失训练的内层瓶颈,以注入用户定义的结构。我们在三个实验中验证其有效性:一、在不牺牲重建质量的前提下,强制潜变量通道有序;二、将潜变量与语义嵌入对齐,并分析其对下游扩散建模的影响;三、引入等变性,确保输入波形的滤波操作直接对应于潜空间中的特定变换。最终,Re-Bottleneck 框架提供了一种灵活高效的方案,可使神经音频模型的表示适应不同应用场景,且只需极少额外训练。

原文摘要 · Abstract (English)

Neural audio codecs and autoencoders have emerged as versatile models for audio compression, transmission, feature-extraction, and latent-space generation. However, a key limitation is that most are trained to maximize reconstruction fidelity, often neglecting the specific latent structure necessary for optimal performance in diverse downstream applications. We propose a simple, post-hoc framework to address this by modifying the bottleneck of a pre-trained autoencoder. Our method introduces a "Re-Bottleneck", an inner bottleneck trained exclusively through latent space losses to instill user-defined structure. We demonstrate the framework's effectiveness in three experiments. First, we enforce an ordering on latent channels without sacrificing reconstruction quality. Second, we align latents with semantic embeddings, analyzing the impact on downstream diffusion modeling. Third, we introduce equivariance, ensuring that a filtering operation on the input waveform directly corresponds to a specific transformation in the latent space. Ultimately, our Re-Bottleneck framework offers a flexible and efficient way to tailor representations of neural audio models, enabling them to seamlessly meet the varied demands of different applications with minimal additional training.

音频编码潜在空间自编码器结构控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。