用先验知识加速结构化生成，速度提升250倍
WGRAMMAR: Leverage Prior Knowledge to Accelerate Structured Decoding
- 将语法约束拆分为静态和动态部分，离线预编译静态结构
- 运行时仅实例化动态参数，减少状态追踪与掩码开销
- 适合需高效生成JSON/HTML等格式的工业级应用
结构化解码使大语言模型能生成符合下游系统要求的输出格式，如HTML或JSON。但现有方法因语法编译、状态跟踪和掩码创建存在效率瓶颈。我们观察到许多实际任务中输出结构蕴含强先验知识。基于此，提出将约束分解为静态与动态组件——离线预编译静态结构，运行时使用语法片段实例化动态参数。不依赖下推自动机，改用组合式操作符建模正则格式，实现更低的转换延迟。我们提出wgrammar，一个轻量级解码引擎，集成领域感知简化、约束分解与掩码缓存，相比现有系统最高提速250倍。代码已公开于https://github.com/wrran/wgrammar。
原文摘要 · Abstract (English)
Structured decoding enables large language models (LLMs) to generate outputs in formats required by downstream systems, such as HTML or JSON. However, existing methods suffer from efficiency bottlenecks due to grammar compilation, state tracking, and mask creation. We observe that many real-world tasks embed strong prior knowledge about output structure. Leveraging this, we propose a decomposition of constraints into static and dynamic components -- precompiling static structures offline and instantiating dynamic arguments at runtime using grammar snippets. Instead of relying on pushdown automata, we employ a compositional set of operators to model regular formats, achieving lower transition latency. We introduce wgrammar, a lightweight decoding engine that integrates domain-aware simplification, constraint decomposition, and mask caching, achieving up to 250x speedup over existing systems. wgrammar's source code is publicly available at https://github.com/wrran/wgrammar.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。