用隐空间均值流实现语音令牌一步生成，速度提升17倍且质量不降。
One-Step Token-to-Waveform Generation with MeanFlow in Latent Space
- 在压缩隐空间中使用均值流，实现单步语音生成
- 相比多步方法，实时因子（RTF）提升17倍，质量几乎无损
- 通过微调策略解决隐空间错配，不增加推理成本
神经音频编解码器是现代基于大语言模型的文本转语音（TTS）及多模态系统的核心。随着低比特率语义编解码器的兴起，语音令牌到波形（Token2Wav）解码器成为决定感知质量和系统效率的关键瓶颈。传统多步流匹配解码器虽质量优越，但因迭代采样导致高推理延迟，造成严重的质量-速度权衡。本文提出一种新型 Token2Wav 架构，在高度压缩的隐空间中应用均值流（MeanFlow），通过建模平均速度而非瞬时速度场，实现真正的单步生成。在隐空间中操作有效缓解了波形级流模型的内存与稳定性问题，相较多步基线实现高达17×的实时因子（RTF）提升，且质量损失可忽略。此外，我们引入优化策略缓解隐空间错配，包括冻结均值流生成器的仅解码器微调和端到端联合微调，显著提升保真度而不增加推理开销。代码与演示已公开。
原文摘要 · Abstract (English)
Neural audio codecs are central to modern LLM-based Text-to-Speech (TTS) and multimodal systems. As low-bitrate semantic codecs gain prominence, the Token-to-Waveform (Token2Wav) decoder becomes a bottleneck determining both perceptual quality and system efficiency. Conventional multi-step flow-matching decoders offer superior quality but suffer from high inference latency due to iterative sampling, creating a severe quality-speed trade-off. In this paper, we propose a novel Token2Wav architecture that overcomes this limitation by applying MeanFlow in a highly compressed latent space. By modeling the average velocity rather than the instantaneous velocity field, MeanFlow enables true one-step generation. Operating in the latent domain mitigates the memory and stability issues of waveform-level flows, yielding up to a 17$\times$ improvement in Real-Time Factor (RTF) compared to multi-step baselines with negligible quality degradation. Furthermore, we introduce refinement strategies that mitigate latent mismatch, including decoder-only fine-tuning with the MeanFlow generator frozen and end-to-end joint fine-tuning, improving fidelity without increasing inference-time cost. Code and demo are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。