用约束推理提升大模型生成内容的准确性
Large Language Model Meets Constraint Propagation
- 将大模型生成与约束规划结合，形成双向约束传播机制
- 在COLLIE基准上显著提升严格约束任务的生成成功率
- 适合需要强内容控制的文本生成场景
大语言模型擅长生成流畅文本，但缺乏对外部约束的强制能力，因其逐词生成且无显式控制。GenCP通过将语言模型预测与约束规划结合，将文本生成建模为约束满足问题。本文改进GenCP，引入掩码语言模型（MLM）进行领域预览，实现双向约束传播，利用前后文信息协同优化。该方法弥合了词级别预测与结构化约束之间的差距，提升了生成结果的可靠性与约束符合度。在COLLIE基准上的评估表明，通过MLM调用提供领域预览可显著改善性能。尽管增加额外的MLM调用和回溯次数，整体仍更高效利用了大模型推理，尤其在强内容约束任务中生成更可行、有意义的结果。
原文摘要 · Abstract (English)
Large Language Models (LLMs) excel at generating fluent text but struggle to enforce external constraints because they generate tokens sequentially without explicit control mechanisms. GenCP addresses this limitation by combining LLM predictions with Constraint Programming (CP) reasoning, formulating text generation as a Constraint Satisfaction Problem (CSP). In this paper, we improve GenCP by integrating Masked Language Models (MLMs) for domain generation, which allows bidirectional constraint propagation that leverages both past and future tokens. This integration bridges the gap between token-level prediction and structured constraint enforcement, leading to more reliable and constraint-aware text generation. Our evaluation on COLLIE benchmarks demonstrates that incorporating domain preview via MLM calls significantly improves GenCP's performance. Although this approach incurs additional MLM calls and, in some cases, increased backtracking, the overall effect is a more efficient use of LLM inferences and an enhanced ability to generate feasible and meaningful solutions, particularly in tasks with strict content constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。