让文本生成一步到位，靠共享隐变量建模词间关联。
Latent-Kernel Discrete Flow Maps for Few-Step Generation
- 用共享隐变量统一多个独立组件，实现跨位置关联建模。
- 单步生成即可捕捉语义耦合，性能超越基线2.1到3.3倍。
- 适合追求高效高质生成的开发者，尤其适配少步生成场景。
离散扩散与流匹配模型通过多步去噪生成序列，但为保持每步计算廉价，通常对位置间转移进行因子分解，独立决定每个词元。这导致在少步生成时难以处理如主谓一致等跨位置依赖关系。现有方法依赖慢速教师模型蒸馏或修正，受限于教师上限。本文提出从头构建的隐变量核离散流映射（LKF），由M个因子化组件构成，通过单一共享隐变量关联。条件于该隐变量，各组件计算廉价；当M较小时，可通过闭式求和合并混合结果。单步生成即可同时分配相关词元，采样复杂度与因子化模型相当——仅需为每条序列抽取一次隐变量并全程复用。我们证明掩码扩散语言模型（MDLM）是LKF在M=1时的特例。在One-Billion-Word（LM1B）与WikiText-103上的实验表明，LKF模型学习到高度异质的组件，生成困惑度相比似然基线提升2.1至3.3倍，且不损失多样性。性能随M增长，当M=8时超越蒸馏与修正型少步采样器。代码已开源：https://github.com/mansoor181/lkf.git
原文摘要 · Abstract (English)
Discrete diffusion and flow-matching models denoise a sequence over many steps, but to keep each step cheap, they factorize the transition across positions and decide every token independently. This makes few-step generation challenging for text when the target couples two positions, such as a subject and a verb that must agree. An independent update commits to them separately, and many function evaluations are spent repairing the mismatch. Existing few-step methods buy back the lost correlation by distilling or rectifying a slow teacher, and so inherit the teacher's quality ceiling. We ask instead whether a model can express correlated steps natively, and answer with Latent-Kernel Discrete Flow Maps (LKF), a from-scratch flow-map kernel that is a mixture of M factorized components tied by a single shared latent. Conditioned on the latent, each component is cheap, and the mixture is summed over the latent in closed form for small M. We show that a single step places mass on correlated completions with the same sampling time complexity as a factorized model, since one latent is drawn per sequence and reused across the entire denoising trajectory. We also show that the Masked Diffusion Language Model (MDLM) is a special case of our LKF model at M=1. The experiments for unconditional text generation on the One-Billion-Word (LM1B) and WikiText-103 benchmarks show that our LKF model learns strongly heterogeneous components and improves generative perplexity by 2.1x to 3.3x over the likelihood baselines without losing diversity. The gain grows with M, and at M=8, it surpasses distilled and rectified few-step samplers. The source code is available at: https://github.com/mansoor181/lkf.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。