让大模型生成JSON时严格遵守字数限制，避免出错或无限输出。
TruncProof: A Guardrail for LLM-based JSON Generation under Token-Length Constraints
- 基于LL(1)解析特性，动态估算完成合法JSON所需的最少词元数。
- 在严格词元约束下仍能生成语法正确的JSON，成功率接近100%。
- 兼容先进解码策略，适合需要稳定机器可读输出的系统集成场景。
基于大模型的机器可读输出（如JSON）生成受到广泛关注，可用于与外部系统集成。然而，现有方法无法严格控制生成的最大词元数，导致无限生成或截断输出，引发系统故障。为此，我们提出TruncProof，一种新型语法约束生成方法，使大模型在预设词元限制下生成语法正确的JSON。通过利用LL(1)解析器的性质，TruncProof在每一步解码中高效估算完成合法输出所需的最小词元数。在Text-to-JSON指令任务上的实验表明，即便在严格词元约束下，TruncProof也能成功生成语法正确的输出。此外，我们证明TruncProof可有效结合先进解码策略，生成不仅语法正确且语义准确的结果。源代码已公开于https://github.com/Yosshi999/TruncProof。
原文摘要 · Abstract (English)
The LLM-based generation of machine-readable outputs such as JSON has attracted significant attention for integration with external systems. However, existing approaches cannot strictly enforce the maximum number of tokens to be generated, leading to infinite generation or truncated outputs that cause a system malfunction. To address this limitation, we propose TruncProof, a novel grammar-constrained generation method that enables LLMs to produce grammatically valid JSONs while adhering to a predefined token limit. By leveraging the properties of LL(1) parsers, TruncProof efficiently approximates the minimum number of tokens required to complete a grammatically valid output at each decoding step. Experiments on the Text-to-JSON instruction tasks demonstrate that TruncProof successfully generates syntactically correct outputs even under strict token constraints. Furthermore, we show that TruncProof can be effectively combined with advanced decoding strategies, resulting in outputs that are not only grammatically valid but also semantically accurate. The source code is public at https://github.com/Yosshi999/TruncProof
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。