用解析器状态修正语法约束生成的偏差,不改模型也能提速提质量。
The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding

- 利用解析器内部状态和候选词做轻量级逻辑修正
- 在多个语法上显著提升生成质量,接近无约束表现
- 适合需要快速高质语法生成的场景,如代码/结构化文本
语法约束解码(GCD)通过每步屏蔽不符合语法的词元来强制语言模型生成语法正确输出。但严格屏蔽会扭曲模型原本的概率分布,常导致生成有效但次优的结果。虽然在线采样可恢复分布,但需昂贵的迭代重采样。我们的关键洞察是:增量解析过程中维护的内部解析器与词法分析器状态,已隐含未来语法合法性信息——正是恢复模型真实分布所需。我们提出一种轻量级、离线训练的逻辑修正方法,基于该语法与词法状态及候选下一词进行调整。由于这些状态已是解析必经步骤,提取几乎无额外开销,且不修改原始模型权重。在多个语法上,该修正显著缩小了屏蔽分布与模型真实分布的差距,持续优于掩码与在线采样。其最轻版本仅依赖候选词本身,仍能匹配或超越基线——因为候选词本身已携带前瞻信息,如同解析器常用前瞻词解决歧义。通过恢复被屏蔽的概率质量,实现了模型概率一致性与语法合规的统一。
原文摘要 · Abstract (English)
Grammar Constrained Decoding (GCD) forces Language Models (LMs) to produce syntactically valid outputs by masking out non-conforming tokens at each step. However, rigid masking distorts the model's underlying probability distribution, often biasing generation toward valid but suboptimal outputs. While online sampling restores this distribution, it requires computationally expensive iterative resampling. As a result, existing methods force a compromise between output quality and inference latency. Our key insight is that the internal parser and lexer states inherently maintained during incremental parsing already encode future grammatical validity -- exactly the information required to restore the LM's true distribution. We propose a lightweight, offline-trained logit correction conditioned on this syntactic and lexical state together with candidate next tokens. Because these states are already computed as a necessary part of incremental parsing for masking, extracting them adds negligible overhead while leaving the base LM's weights completely untouched. Across several grammars, this correction substantially closes the gap between the masked distribution and the LM's true distribution, consistently outperforming both masking and online sampling. Even its lightest variant, which relies on the candidate next token alone, still matches or exceeds both baselines: the next token itself carries an implicit lookahead, much like how parsers commonly use a lookahead token to resolve ambiguous decisions. By restoring the probability mass that masking removes, it reconciles the LM's probabilistic integrity with grammar conformance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。