XGrammar-2让大模型智能体动态生成更高效，编译快6倍以上。
XGrammar-2: Dynamic and Efficient Structured Generation Engine for Agentic LLMs
- 用标签触发结构切换，支持复杂动态输出
- 跨请求复用子结构缓存，编译速度超前代6倍
- 适合需要频繁调用工具的智能体系统
现代大模型智能体越来越多依赖动态结构化生成，如工具调用和响应协议。与传统静态结构不同，这类任务在请求间和请求内均存在显著变化，对现有生成引擎提出新挑战。本文提出XGrammar-2，一种面向动态智能体工作负载的结构化生成引擎。设计基于两大核心思想：对标签触发的结构切换提供一等支持，以及在不同输出结构间实现细粒度复用。具体地，XGrammar-2引入TagDispatch实现动态结构分发，Cross-Grammar Cache实现跨语法的子结构级缓存复用。此外，通过基于Earley算法的自适应令牌掩码缓存、即时编译和重复状态压缩进一步提升效率。实验表明，XGrammar-2编译速度比之前引擎快超过6倍，在现代大模型服务系统中端到端开销几乎为零。
原文摘要 · Abstract (English)
Modern LLM agents increasingly rely on dynamic structured generation, such as tool calling and response protocols. Unlike traditional structured generation with static structures, these workloads vary both across requests and within a request, posing new challenges to existing engines. We present XGrammar-2, a structured generation engine for dynamic agentic workloads. Our design is based on two key ideas: first-class support for tag-triggered structure switching, and fine-grained reuse across requests with different output structures. Concretely, XGrammar-2 introduces TagDispatch for dynamic structural dispatching and Cross-Grammar Cache for substructure-level cache reuse across grammars. It further improves efficiency with an Earley-based adaptive token mask cache, just-in-time compilation, and repetition state compression. Experiments show that XGrammar-2 achieves over 6x faster compilation than prior structured generation engines, and incurs near-zero end-to-end overhead in modern LLM serving systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。