短思考反而更准：研究发现语言代理思考越长越容易出错。
Brief Is Better: Non-Monotonic Chain-of-Thought Budget Effects in Function-Calling Language Agents
- 用简短思维链（32词）做函数选择，准确率提升45%
- 超过256词的长思维链导致准确率跌至25%，低于不思考
- 提出结构化简短思维链，可避免函数幻觉且无需调参
语言代理在行动前应思考多久？尽管思维链（CoT）常被认为能提升性能，但在结构化工具使用场景中，其推理长度与准确性的关系仍不清晰。我们系统研究了不同推理预算对函数调用代理的影响，覆盖0到512个词的六种预算，在伯克利函数调用排行榜v3 Multiple的200个任务上进行测试。核心发现是：Qwen2.5-1.5B-Instruct模型呈现显著非单调模式——32词简短推理使准确率从44.0%提升至64.0%（相对提高45%），而256词长推理则降至25.0%（低于无CoT基线，McNemar p < 0.001）。三类错误分解显示，30.5%的任务因选错函数失败；简短CoT将其降至1.5%，起到函数路由作用；长推理反而恶化，28.0%选错函数，18.0%产生幻觉函数。溯源分析表明，88.6%可解任务只需≤32词，平均27.6词，最优范围为8–16词。基于此，我们提出函数路由思维链（FR-CoT），将推理模板化为“Function: [name] / Key args: [...]”，强制初始阶段确定有效函数名。该方法在准确率上等同于自由形式32词推理，同时将函数幻觉降为0.0%，提供结构可靠性保障且无需预算调整。
原文摘要 · Abstract (English)
How much should a language agent think before taking action? Chain-of-thought (CoT) reasoning is widely assumed to improve agent performance, but the relationship between reasoning length and accuracy in structured tool-use settings remains poorly understood. We present a systematic study of CoT budget effects on function-calling agents, sweeping six token budgets (0--512) across 200 tasks from the Berkeley Function Calling Leaderboard v3 Multiple benchmark. Our central finding is a striking non-monotonic pattern on Qwen2.5-1.5B-Instruct: brief reasoning (32 tokens) dramatically improves accuracy by 45% relative over direct answers, from 44.0% to 64.0%, while extended reasoning (256 tokens) degrades performance well below the no-CoT baseline, to 25.0% (McNemar p < 0.001). A three-way error decomposition reveals the mechanism. At d = 0, 30.5% of tasks fail because the model selects the wrong function from the candidate set; brief CoT reduces this to 1.5%, effectively acting as a function-routing step, while long CoT reverses the gain, yielding 28.0% wrong selections and 18.0% hallucinated functions at d = 256. Oracle analysis shows that 88.6% of solvable tasks require at most 32 reasoning tokens, with an average of 27.6 tokens, and a finer-grained sweep indicates that the true optimum lies at 8--16 tokens. Motivated by this routing effect, we propose Function-Routing CoT (FR-CoT), a structured brief-CoT method that templates the reasoning phase as "Function: [name] / Key args: [...]," forcing commitment to a valid function name at the start of reasoning. FR-CoT achieves accuracy statistically equivalent to free-form d = 32 CoT while reducing function hallucination to 0.0%, providing a structural reliability guarantee without budget tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。