提升语法约束生成效率，预处理速度提升17.7倍
Flexible and Efficient Grammar-Constrained Decoding
- 设计新算法，高效对齐分词器与文法的令牌映射
- 离线预处理时间比现有方法快17.71倍，线上掩码计算仍高效
- 适合需要快速生成代码或结构化数据的场景
大型语言模型常需生成符合严格语法规则的结构化输出，如代码片段或格式化数据。语法约束解码（GCD）通过屏蔽会引发非法输出的标记，确保生成结果符合指定上下文无关文法（CFG）。为保证正确性，现有GCD算法需精确计算语言模型子词分词器与文法标记间的对齐关系，并据此生成标记掩码。这一过程效率低下，常见文法的预处理耗时可达数十分钟。本文提出一种新型GCD算法及其实现，离线预处理速度比现有方法快17.71倍，同时保持在线掩码计算的先进效率。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are often asked to generate structured outputs that obey precise syntactic rules, such as code snippets or formatted data. Grammar-constrained decoding (GCD) can guarantee that LLM outputs matches such rules by masking out tokens that will provably lead to outputs that do not belong to a specified context-free grammar (CFG). To guarantee soundness, GCD algorithms have to compute how a given LLM subword tokenizer can align with the tokens used by a given context-free grammar and compute token masks based on this information. Doing so efficiently is challenging and existing GCD algorithms require tens of minutes to preprocess common grammars. We present a new GCD algorithm together with an implementation that offers 17.71x faster offline preprocessing than existing approaches while preserving state-of-the-art efficiency in online mask computation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。