让语言模型先‘思考’再生成，提升逻辑与连贯性。
Stop-Think-AutoRegress: Language Modeling with Latent Diffusion Planning
- 用扩散模型在连续空间中预先规划语义，再生成文本。
- 在叙事连贯性和常识推理上胜过同类模型超70%。
- 无需重训即可控制文本风格，流畅度更好。
停-思-自回归语言扩散模型（STAR-LDM）将潜在扩散规划与自回归生成结合。不同于传统逐标记生成的语言模型，STAR-LDM引入‘思考’阶段，在生成前通过扩散过程在连续空间中优化语义计划，实现全局规划。评估显示，该模型在语言理解基准上显著优于同规模模型，在大模型评判的叙事连贯性和常识推理任务中达到超过70%的胜率。其架构支持轻量分类器直接控制文本属性，无需重新训练，同时在流畅性与控制精度之间表现更优。
原文摘要 · Abstract (English)
The Stop-Think-AutoRegress Language Diffusion Model (STAR-LDM) integrates latent diffusion planning with autoregressive generation. Unlike conventional autoregressive language models limited to token-by-token decisions, STAR-LDM incorporates a "thinking" phase that pauses generation to refine a semantic plan through diffusion before continuing. This enables global planning in continuous space prior to committing to discrete tokens. Evaluations show STAR-LDM significantly outperforms similar-sized models on language understanding benchmarks and achieves $>70\%$ win rates in LLM-as-judge comparisons for narrative coherence and commonsense reasoning. The architecture also allows straightforward control through lightweight classifiers, enabling fine-grained steering of attributes without model retraining while maintaining better fluency-control trade-offs than specialized approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。