TOON比JSON更省token,但需权衡提示词开销。
Token-Oriented Object Notation vs JSON: A Benchmark of Plain and Constrained Decoding Generation
- 用一次性提示学习生成TOON格式,减少冗余结构
- 简单结构下约束解码最省token,TOON优势不明显
- TOON真正高效需在复杂结构中累积语法节省
近期提出的面向标记的对象表示(TOON)旨在以更低的token消耗替代JSON作为向大模型传递结构化数据的序列化格式。尽管在模型理解上表现良好,但缺乏与JSON生成的对比测试。尽管训练数据中未包含TOON语法,其结构简单,暗示可通过单次上下文学习实现准确生成。提示词开销虽不可避免,但可被更短的生成结果所抵消。为此,我们设计基准测试,涵盖结构复杂度、验证流程,并比较纯JSON生成、结构化输出(约束解码)的JSON生成,以及基于单次上下文学习的TOON生成。其中,结构化输出用于建立最低的token预算基线,也为未来测试TOON约束解码推理提供起点。关键发现:在同域生成任务中,TOON表现出有前景的准确率/令牌消耗比,但该优势常被短上下文中的指令开销削弱;纯JSON生成在一次性和最终准确性上表现最佳,甚至优于约束解码的结构化输出,后者仅在降低令牌使用上有优势,整体准确率略有下降,部分模型显著退化。值得注意的是,在简单结构下,约束解码的最低令牌使用量甚至超过TOON,暗示通过xgrammar等框架强制执行TOON可能无法达到预期效果。此外,结果支持一种扩展假设:TOON的真实效率潜力可能呈非线性曲线,仅在特定阈值以上,累积的语法节省才能抵消初始提示开销。
原文摘要 · Abstract (English)
Recently presented Token-Oriented Object Notation (TOON) aims to replace JSON as a serialization format for passing structured data to LLMs with significantly reduced token usage. While showing solid accuracy in LLM comprehension, there is a lack of tests against JSON generation. Though never present in training data, TOON syntax is simple enough to suggest one-shot in-context learning could support accurate generation. The inevitable prompt overhead can be an acceptable trade-off for shorter completions. To test this, we conducted a benchmark creating several test cases with regard to structural complexity, a validation pipeline, and comparing plain JSON generation vs structured output (via constrained decoding) JSON generation vs TOON one-shot in-context learning generation. JSON structured output was included to establish a minimum token budget baseline and to set a starting point for future experiments testing TOON constrained decoding inference enforcement. Key findings: TOON shows promising accuracy/token consumption ratio for in-domain generation tasks, though this advantage is often reduced by the "prompt tax" of instructional overhead in shorter contexts. Plain JSON generation shows the best one-shot and final accuracy, even compared with constrained decoding structured output, where the only significant advantage is the lowest token usage as a trade-off for slightly decreased accuracy overall and significant degradation for some models. Notably, for simple structures, this "lowest token usage" of constrained decoding outperformed even TOON, hinting that TOON enforcing via frameworks such as xgrammar may not yield the desired results. Furthermore, the results suggest a scaling hypothesis: TOON's true efficiency potential likely follows a non-linear curve, shining only beyond a specific point where cumulative syntax savings amortize the initial prompt overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。