用后验精修加速语言生成，32倍少步数仍保高质量。
Posterior Refinement: Fast Language Generation via Any-Order Flow Maps
- 通过掩码式噪声调度，让模型单步生成全序列并评估每个词的合理性。
- 在多个基准上实现与离散基线相当质量，仅需32倍更少的积分步数。
- 适合追求高速高质生成的场景，尤其适合部署在资源受限环境。
非自回归生成虽可迭代优化任意词元子集，但现有方法受限。掩码扩散模型（MDMs）因因子化误差导致多词同时生成时质量下降；流映射语言模型（FMLMs）虽通过联合序列传输实现优秀少步生成，却牺牲了MDMs的推理灵活性。本文提出FMLM+，引入掩码式噪声调度，使模型在单步生成完整序列的同时，后验评估每个词的全局一致性。基于此，我们提出后验精修策略，支持模型自适应修正输出，在仅32倍更少积分步数（NFEs）下达到离散基线性能。在多个基准测试中，FMLM+结合后验精修显著优于MDM和FMLM家族，为高保真语言建模提供可扩展的高效基础。
原文摘要 · Abstract (English)
Non-autoregressive generation offers a powerful paradigm for iterative refinement, allowing models to recursively critique, erase and regenerate arbitrary subsets of tokens. However, existing non-autoregressive models fail to realize this potential. Masked Diffusion Models (MDMs) suffer from factorization error, causing sample quality to collapse when generating multiple tokens simultaneously. Flow Map Language Models (FMLMs) circumvent this bottleneck via joint sequence transport for excellent few-step generation, but sacrifice the inference-time flexibility of MDMs. We introduce FMLM+, a framework that bridges this gap by equipping FMLM with masking-style noise schedules. While generating the full sequence in a single step, FMLM+ simultaneously scores the global consistency of each token a posteriori. We leverage this to introduce Posterior Refinement, a novel inference-time refinement strategy that enables the model to adaptively self-correct its outputs, matching the performance of discrete baselines with 32x fewer NFEs. Across diverse benchmarks, we demonstrate that FMLM+ with Posterior Refinement improves the speed--quality tradeoff over both MDM and FMLM families, providing a scalable foundation for high-fidelity language modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。