通过目标KL正则化调控音频自编码器压缩率，提升音效生成质量。
Taming Audio VAEs via Target-KL Regularization
- 用目标KL正则化控制音频自编码器的压缩比特率
- 实现不同压缩率下的音质-速率权衡，构建率失真曲线
- 帮助找到文本转声音生成的最佳压缩设置
潜在扩散模型已成为音频生成等任务的主流范式，如文本到音频、文本到音乐和文本到语音。其关键组件是自编码器（VAE），将高维信号压缩为低帧率连续表示，便于后续预测。然而，对这些VAE进行正则化颇具挑战：过度正则化导致输出质量差，而正则化不足则难以预测。本文提出一种通过压缩研究该权衡的框架，利用目标KL正则化在特定比特率下训练音频VAE。这使得可直接与成熟的离散神经音频编解码模型对比，并构建音频VAE的率失真曲线。我们评估了目标KL正则化对文本转声音生成的影响，发现扫描不同压缩率有助于识别最优生成配置。
原文摘要 · Abstract (English)
Latent diffusion models have emerged as the dominant paradigm for many generation tasks including audio generation such as text-to-audio, text-to-music and text-to-speech. A key component of latent diffusion is an autoencoder (VAE) that compresses high-dimensional signals into a low frame rate continuous representation that is conducive for downstream prediction. Regularizing these VAEs is challenging, as there is a trade-off between over-regularized (poor output quality) and under-regularized (difficult to predict) latent representations. We propose a framework for studying this trade-off through compression and train Audio VAEs at specific bitrates via target-KL regularization. This allows direct comparison to well-studied discrete neural audio codec models, and the construction of rate-distortion curves for audio VAEs. We evaluate the impact of target-KL regularization on text-to-sound generation and find that sweeping compression rates is helpful in identifying the optimal generation setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。