让扩散语言模型生成符合语法的文本，无需额外训练。
Continuous Diffusion Models Can Obey Formal Syntax
- 用正则表达式定义语法约束，通过解析得分引导采样
- 在180个约束下满足率68%-96%，困惑度仅小幅上升
- 适用于需严格格式输出的场景，如生成合法JSON
扩散语言模型因其全局非因果生成机制，是自回归模型的有力替代，但其连续潜在空间难以施加离散约束（如输出需符合指定JSON Schema）。本文提出一种无需训练的引导方法，通过构建解析得分函数估算潜在状态解码为符合给定正则表达式的字符串的概率，并利用梯度引导采样过程，不依赖辅助分类器。去噪过程基于语法有效性条件下的基础模型。我们在PLAID扩散模型基础上实现Diffinity，在覆盖JSON与自然语言的180个正则表达式约束上进行评估。结果表明，Diffinity在保持低困惑度代价的同时，达成68%-96%的约束满足率，优于自回归约束解码在约束满足率与输出质量上的表现。代码已开源：github.com/large-loris-models/Diffinity。
原文摘要 · Abstract (English)
Diffusion language models offer a promising alternative to autoregressive models due to their global, non-causal generation process, but their continuous latent dynamics make discrete constraints -- e.g., the output should be a JSON file that matches a given schema -- difficult to impose. We introduce a training-free guidance method for steering continuous diffusion language models to satisfy formal syntactic constraints expressed using regular expressions. Our approach constructs an analytic score estimating the probability that a latent state decodes to a valid string accepted by a given regular expression, and uses its gradient to guide sampling, without training auxiliary classifiers. The denoising process targets the base model conditioned on syntactic validity. We implement our method in Diffinity on top of the PLAID diffusion model and evaluate it on 180 regular-expression constraints over JSON and natural-language benchmarks. Diffinity achieves 68-96\% constraint satisfaction while incurring only a small perplexity cost relative to unconstrained sampling, outperforming autoregressive constrained decoding in both constraint satisfaction and output quality. Diffinity is open-sourced at github.com/large-loris-models/Diffinity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。