FlowMAC用条件流匹配实现3kbps超低码率高质量音频编码
FlowMAC: Conditional Flow Matching for Audio Coding at Low Bit Rates
- 基于条件流匹配联合训练声谱编码器、量化器与解码器
- 3kbps下主观质量媲美其他方法6kbps的水平，且支持实时CPU编码
- 可调节推理复杂度，适合资源受限场景的高保真音频应用
本文提出FlowMAC，一种基于条件流匹配（CFM）的新型神经音频编解码器，可在极低码率下实现高质量通用音频压缩。FlowMAC联合学习梅尔频谱编码器、量化器与解码器。推理时，解码器通过微分方程求解器集成连续归一化流，生成高质量梅尔频谱图。这是首次将基于CFM的方法应用于通用音频编码，实现了可扩展、简洁且内存高效的训练。主观评估显示，FlowMAC在3 kbps下的音质与现有最先进的基于GAN和DDPM的神经音频编解码器在6 kbps时相当。此外，其可调推理流程支持在复杂度与质量间灵活权衡，可在CPU上实现实时编码，同时保持高感知质量。
原文摘要 · Abstract (English)
This paper introduces FlowMAC, a novel neural audio codec for high-quality general audio compression at low bit rates based on conditional flow matching (CFM). FlowMAC jointly learns a mel spectrogram encoder, quantizer and decoder. At inference time the decoder integrates a continuous normalizing flow via an ODE solver to generate a high-quality mel spectrogram. This is the first time that a CFM-based approach is applied to general audio coding, enabling a scalable, simple and memory efficient training. Our subjective evaluations show that FlowMAC at 3 kbps achieves similar quality as state-of-the-art GAN-based and DDPM-based neural audio codecs at double the bit rate. Moreover, FlowMAC offers a tunable inference pipeline, which permits to trade off complexity and quality. This enables real-time coding on CPU, while maintaining high perceptual quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。