让大模型生成时可回溯修正，提升结构化输出准确性
IterGen: Iterative Semantic-aware Structured LLM Generation with Backtracking
- 支持前后移动修改生成内容，基于语法符号定位与缓存复用
- 在隐私泄露减少和SQL/vega-lite生成上准确率显著提升
- 适合需要高精度结构化输出的开发者与研究者
大型语言模型在自然语言和代码生成中广泛应用，但其输出常出现幻觉、毒性及错误。现有结构化生成工具采用自左至右解码且不支持回溯，难以在生成过程中修正错误。为此，我们提出IterGen，一个用户友好的迭代式语法引导生成库，允许基于语法符号在生成结果中双向移动。通过符号到位置的映射及键值缓存状态保持，IterGen实现高效结构化生成并支持过程修正。我们在两个关键应用中验证其有效性：减少大模型输出中的隐私泄露，以及提升生成SQL和Vega-Lite查询的准确性。代码与资源详见https://structuredllm.com。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are widely used for tasks such as natural language and code generation, but their outputs often suffer from issues like hallucination, toxicity, and incorrect results. Current libraries for structured LLM generation rely on left-to-right decoding without support for backtracking, limiting the ability to correct or refine outputs mid-generation. To address this, we introduce IterGen, a user-friendly library for iterative, grammar-guided LLM generation that enables users to move both forward and backward within the generated output based on grammar symbols. By leveraging a symbol-to-position mapping and maintaining the key-value (KV) cache state, IterGen ensures efficient and structured generation while allowing for corrections during the process. We demonstrate IterGen's effectiveness in two important applications: reducing privacy leakage in LLM outputs and improving the accuracy of LLM-generated SQL and Vega-Lite queries. Our code and additional resources are available at https://structuredllm.com.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。