arXiv:2607.11738eess.AS2026-07被引 2

高效压缩音频的生成模型,32分钟音频仅需541毫秒编码。

Qwen-Audio-VAE Technical Report

论文配图:Qwen-Audio-VAE Technical Report
图 1 · 摘自论文原文
  • 采用因果结构与窗口Transformer,兼顾重建质量与压缩效率。
  • 在500万小时多领域音频上训练,支持跨域音频高质量还原。
  • 适合大规模文本到音频生成,特别适合对速度要求高的场景。

我们提出 extbf{Qwen-Audio-VAE},一种低比特率、快速编码的连续音频自编码器,专为可扩展的通用音频生成设计。该模型基于一个核心原则:音频自编码器不仅要高保真地重建多样音频,还需生成足够紧凑的潜在表示,以支持大规模文本到音频训练。Qwen-Audio-VAE结合因果编码器-解码器、窗口Transformer模块和多判别器训练,在重建质量与压缩率之间取得良好平衡。模型在500万小时多领域音频上大规模训练,实现对异构声学条件下的鲁棒重建。为提升计算效率,采用非对称编码器-解码器架构,并引入延迟感知编码器剪枝,最大化编码吞吐量。公开语音、音乐和声音重建基准测试表明,Qwen-Audio-VAE在多种音频领域具有优异泛化能力,尤其高效:仅需541毫秒即可编码32分钟音频。总体而言,Qwen-Audio-VAE为高效通用音频生成提供了一个高质量、紧凑且高吞吐的表示骨干。

原文摘要 · Abstract (English)

We introduce \textbf{Qwen-Audio-VAE}, a suite of low-bitrate, fast-encoding continuous audio autoencoders designed for scalable general audio generation. The model is built around a simple but important principle: an audio VAE should not only reconstruct diverse audio with high fidelity, but also produce compact latent representations fast enough to support large-scale text-to-audio training. Qwen-Audio-VAE combines a causal encoder-decoder, window Transformer blocks, and multi-discriminator training to achieve a strong balance between reconstruction quality and compression rate. The model is trained at scale on 5 million hours of multi-domain audio, enabling robust reconstruction across heterogeneous acoustic conditions. To further improve computational efficiency, we adopt an asymmetric encoder-decoder backbone and introduce latency-aware encoder pruning to maximize encoding throughput. Experiments on public speech, music, and sound reconstruction benchmarks show that Qwen-Audio-VAE generalizes well across diverse audio domains and is particularly efficient, requiring only 541 ms to encode 32 minutes of audio. Overall, Qwen-Audio-VAE provides a high-quality, compact, and high-throughput representation backbone for efficient general audio generation.

音频生成自编码器高效编码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。