用迭代掩码并行解码，实现高音质音频生成的快速推理。
IMPACT: Iterative Mask-based Parallel Decoding for Text-to-Audio Generation with Diffusion Modeling
- 在连续潜空间中结合扩散模型与迭代掩码并行解码。
- 在AudioCaps上达到最优的FD和FAD指标，延迟显著降低。
- 适合追求高音质且需要低延迟生成的应用场景。
文本到音频生成能根据自然语言提示合成逼真的声音或音乐。基于扩散的框架（如Tango和AudioLDM系列）是当前该领域的最先进方法，尽管音频保真度高，但因扩散采样过程缓慢导致推理延迟大。MAGNET通过离散标记的掩码机制实现并行解码，提升了速度，但音质仍逊于扩散模型。本文提出IMPACT，一种在连续潜空间中利用扩散建模进行迭代掩码并行解码的文本到音频生成框架，在保持高速推理的同时显著提升音质。在AudioCaps数据集上的实验表明，IMPACT在关键指标如弗雷歇距离（FD）和弗雷歇音频距离（FAD）上达到当前最优水平，同时显著降低推理延迟。
原文摘要 · Abstract (English)
Text-to-audio generation synthesizes realistic sounds or music given a natural language prompt. Diffusion-based frameworks, including the Tango and the AudioLDM series, represent the state-of-the-art in text-to-audio generation. Despite achieving high audio fidelity, they incur significant inference latency due to the slow diffusion sampling process. MAGNET, a mask-based model operating on discrete tokens, addresses slow inference through iterative mask-based parallel decoding. However, its audio quality still lags behind that of diffusion-based models. In this work, we introduce IMPACT, a text-to-audio generation framework that achieves high performance in audio quality and fidelity while ensuring fast inference. IMPACT utilizes iterative mask-based parallel decoding in a continuous latent space powered by diffusion modeling. This approach eliminates the fidelity constraints of discrete tokens while maintaining competitive inference speed. Results on AudioCaps demonstrate that IMPACT achieves state-of-the-art performance on key metrics including Fréchet Distance (FD) and Fréchet Audio Distance (FAD) while significantly reducing latency compared to prior models. The project website is available at https://audio-impact.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。