给大模型加推理结构，超了令牌预算就更准。
Thinking Costs Tokens: When More Structure is Worth the Price
- 用规划+验证+修复架构替代单次调用
- 1500令牌以上时准确率领先单次调用约4个百分点
- 适合需要高可靠性的金融问答任务
为语言模型增加推理结构可实现搜索、验证和修正，但这些操作会消耗本应高效利用的令牌预算。本文研究是否存在一个令牌预算阈值：低于该阈值时，规划与验证的开销会损害性能；高于该阈值时则带来收益。我们在FinQA和TAT-QA金融推理任务上评估了两种系统，使用GPT-5.4 mini在14个预算层级（250至42,000输出等价令牌）下进行测试。第一种为单次调用的单体系统，第二种是加入规划、无标签检查和修复能力的验证搜索架构。共完成1000例，总计28,000个有效结果。在最低两个层级，两者均得0%准确率，因无法容纳完整提示。在1,000令牌时，单体系统达18%准确率，而验证搜索接近0%，因规划开销挤占了回答空间。从1,500令牌起，验证搜索超越单体系统并持续领先，在最高层级达到约44%，而单体系统仅达约40%。交叉点位于1,000至1,500令牌之间，经严格交集-并集检验确认（两端p ≤ 0.001）。
原文摘要 · Abstract (English)
Adding inference structure to a language model lets it search, verify, and revise, but these actions consume the very budget they are supposed to use well. In this paper, we investigate whether there exists a token-budget threshold, below which the overhead of planning and verification hurts performance and above which it helps. We evaluate two systems on FinQA and TAT-QA financial reasoning tasks, using GPT-5.4 mini across 14 budget tiers ranging from 250 to 42,000 output-equivalent tokens. The first system is a monolith, which is a single LLM call. The second is a verified search architecture that adds planning, label-blind checking, and repair capabilities. We run 1,000 cases for a total of 28,000 completed cells. Both systems score 0% at the two lowest tiers, where neither can fit a complete prompt. At 1,000 tokens, the monolith reaches 18% accuracy while verified search scores near 0%, since the planning overhead leaves no room for an answer. From 1,500 tokens onward, verified search surpasses the monolith and maintains a consistent advantage, reaching approximately 44% at the highest tiers while the monolith reaches approximately 40%. The crossover occurs between 1,000 and 1,500 output-equivalent tokens, confirmed by a strict intersection-union test ($p \le 0.001$ at both endpoints).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。