将连续扩散模型用于文本生成，提升速度与连贯性。
Towards Latent Diffusion Suitable For Text
- 提出神经流扩散模型，适配离散文本空间
- 在相同规模下，生成质量接近自回归模型
- 适合追求高效高质量文本生成的研究者
语言扩散模型旨在提升生成速度和连贯性，优于自回归大语言模型。本文提出神经流扩散模型（NFDM），扩展了原有方法，使连续扩散模型可直接应用于离散状态空间。NFDM从数据中学习多变量前向过程，确保前向过程与生成轨迹契合语言建模需求。实验表明，该模型显著缩小了与同规模自回归模型的似然差距，同时生成样本质量达到先前潜变量扩散模型水平。
原文摘要 · Abstract (English)
Language diffusion models aim to improve sampling speed and coherence over autoregressive LLMs. We introduce Neural Flow Diffusion Models for language generation, an extension of NFDM that enables the straightforward application of continuous diffusion models to discrete state spaces. NFDM learns a multivariate forward process from the data, ensuring that the forward process and generative trajectory are a good fit for language modeling. Our model substantially reduces the likelihood gap with autoregressive models of the same size, while achieving sample quality comparable to that of previous latent diffusion models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。