用前缀树自动机加速大集合约束生成,速度提升29倍。
Trie Automata for Constrained Decoding over Large Finite Sets
- 基于前缀树和多模式匹配预计算每步有效词元掩码
- 批量服务下吞吐达219请求/秒,较XGrammar快29倍
- 支持超大词表(10000个选项)且保证输出100%合法
大型语言模型越来越多需要生成符合预定义结构的输出,常见约束是限定在有限的有效字符串集中选择。现有约束解码系统依赖通用语法编译,当有效值数量达数千时效率急剧下降,形成基数瓶颈。本文提出前缀树自动机,利用有限集的共用前缀、有限深度和已知基数特性,通过Aho-Corasick多模式匹配预先计算节点级词元掩码。该方法相比vLLM和SGLang中主要后端XGrammar,每步有效词元计算速度提升7倍(0.65微秒 vs 5.8微秒),在K ≥ 300时编译速度提升2–6.5倍。由于预计算掩码可实现无状态服务路径,跳过引导解码流程,该优势在批量服务中进一步放大:批量大小为256时,端到端vLLM吞吐达219请求/秒,相较XGrammar的7.5请求/秒提升29倍。这一29倍提升结合了算法加速与路径优化,仅预计算掩码才能实现。在七个分词器家族(32K–262K词汇量)中,前缀树可在K = 10,000时保持编译时间低于100毫秒,且每步开销不随集合规模增加,同时确保100%输出合法性。
原文摘要 · Abstract (English)
Large language models increasingly need to generate structured outputs that conform to predefined schemas, with one common constraint being selection from a finite set of valid strings. Current constrained decoding systems handle this through general-purpose grammar compilation, which becomes prohibitively slow as the number of valid values grows into the thousands, a cardinality wall. We introduce the trie automaton, a specialized mechanism that exploits finite-set structure (shared prefixes, bounded depth, known cardinality) via Aho-Corasick multi-pattern matching to precompute per-node token masks. The trie achieves 7X faster per-step valid-token computation (0.65 us vs. 5.8 us) compared to XGrammar, one of the primary backends in vLLM and SGLang, and 2--6.5X faster compilation at K >= 300. Because precomputed masks enable a stateless serving path that bypasses the guided decoding pipeline, this advantage compounds in batch serving: end-to-end vLLM throughput reaches 219 req/s vs. XGrammar's 7.5 req/s at batch size 256 (29X). The 29X combines the algorithmic speedup with integration-path savings that only precomputed masks can unlock. Across seven tokenizer families (32K--262K vocabulary), the trie maintains sub-100ms compilation up to K = 10,000 and flat per-step cost regardless of set size, while guaranteeing 100% output validity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。