arXiv:2506.00385cs.SDcs.AI2025-06被引 9

MagiCodec通过注入高斯噪声提升音频编码的语义表达,兼顾高保真重建与生成能力。

MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generation

  • 单层流式Transformer架构,多阶段训练中引入高斯噪声和隐变量正则化
  • 在语音重建与下游生成任务上均优于现有方法,编码符符合语言类的齐普夫分布
  • 适合基于语言模型的音频生成研究者使用,尤其关注编码语义表达

神经音频编解码器在高效将原始音频波形映射为离散标记表示方面取得了显著进展,这是现代音频生成模型的基础。然而,大多数现有编解码器主要优化重建质量,常牺牲编码标记的下游可用性。为克服这一瓶颈,我们提出新型单层、流式Transformer-based音频编解码器MagiCodec。其采用多阶段训练流程,结合高斯噪声注入与潜在变量正则化,明确提升生成代码的语义表达力,同时保持高重建保真度。我们从理论上推导了频域中噪声注入的影响,证明其能有效抑制高频成分并促进鲁棒标记化。大量实验表明,MagiCodec在重建质量和下游任务上均超越现有最佳方法。值得注意的是,MagiCodec生成的标记呈现出类似自然语言的齐普夫分布,从而增强与基于语言模型的生成架构的兼容性。代码与预训练模型已公开于https://github.com/Ereboas/MagiCodec。

原文摘要 · Abstract (English)

Neural audio codecs have made significant strides in efficiently mapping raw audio waveforms into discrete token representations, which are foundational for contemporary audio generative models. However, most existing codecs are optimized primarily for reconstruction quality, often at the expense of the downstream modelability of the encoded tokens. Motivated by the need to overcome this bottleneck, we introduce $\textbf{MagiCodec}$, a novel single-layer, streaming Transformer-based audio codec. MagiCodec is designed with a multistage training pipeline that incorporates Gaussian noise injection and latent regularization, explicitly targeting the enhancement of semantic expressiveness in the generated codes while preserving high reconstruction fidelity. We analytically derive the effect of noise injection in the frequency domain, demonstrating its efficacy in attenuating high-frequency components and fostering robust tokenization. Extensive experimental evaluations show that MagiCodec surpasses state-of-the-art codecs in both reconstruction quality and downstream tasks. Notably, the tokens produced by MagiCodec exhibit Zipf-like distributions, as observed in natural languages, thereby improving compatibility with language-model-based generative architectures. The code and pre-trained models are available at https://github.com/Ereboas/MagiCodec.

音频生成编解码器语言模型高斯噪声

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。