模型能生成正确调用，但接口会悄悄过滤掉，导致评估结果失真。
Interface-Induced Trajectory Censoring
- 接口在模型输出后主动屏蔽调用轨迹，造成评估误差。
- 同一模型在不同接口下工具调用成功率从0飙升至636次/115任务。
- 该问题存在于接口契约交互中，修复单边接口无效。
模型评估中的工具调用率由服务端栈读取，但即使模型生成了格式正确的调用，接口仍可能在下游可见前将其屏蔽。在BFCL v4数据上，相同模型使用不同服务适配器，得分可从0.00变为0.96/0.19。2×2实验显示，主效应均为零，全部差异源于交互项——无组件缺陷，修复任一端口均无效。在tau-bench的115个交互零售任务中，同一次切换使服务端解析调用数从0增至636，成功执行任意工具的任务数从0增至103。对Qwen2.5-Coder进行21倍规模测试，服务端始终解析0/100，而模型发出的有效调用达80/100（32B，校准后约72）。在匹配环境下，沉默失败比例稳定在0-2，该预测提前提交至仓库。Llama-3.1-8B在严格模式下，调用任务函数的比例从23%降至0。该不匹配贯穿训练循环，且具规模依赖性：verl的AgentLoop在7B时，115次生成中有45次含完整调用，但0次被接受、0次执行、0次返回观测；1.5B时同样为0，故分别报告。评估时修复适配器仅恢复机制，未显著提升效果：解析率0→84，救援任务0→9，通过率53→62（不显著）。我们发布一个98行预检脚本，可捕获所有此类静默失败。观察到的工具调用率并非模型本身属性，而是模型-接口栈共同决定的结果。
原文摘要 · Abstract (English)
Agent evaluations report a tool-call rate read off the serving stack. That number can be zero while the model is emitting well-formed calls: the interface censors the trajectory before anything downstream sees it. On BFCL v4's own data, executor and scorer, holding weights, cases, decoding and seeds fixed and changing only the serving adapter, the same model scores 0.00 or 0.96 / 0.19. A 2x2 over chat template and parser locates the effect exactly: both main effects are exactly zero and all of it sits in the interaction -- no component is defective, and repairing one side of the contract buys precisely nothing. On tau-bench's 115 interactive retail tasks the same swap moves server-parsed calls from 0 to 636 and tasks reaching any tool execution from 0 to 103. Our probe reproduces the funnel across a 21x scale range of Qwen2.5-Coder: the server parses 0/100 at every size while well-formed emitted calls rise to 80/100 at 32B (~72 after calibration against an adjudicated gold standard). Under a matched envelope, across a comparable scale span, the silent fraction stays at 0-2, a prediction committed to the repository before the run. Llama-3.1-8B's 23% rate of calling the task function itself as a tool falls to 0 under one strict:true flag. The mismatch reaches inside the training loop, and its consequence is scale-dependent: in verl's AgentLoop at 7B, 45 of 115 generations carry a complete call; 0 are accepted, 0 execute, 0 return an observation. At 1.5B the same zero is over-determined, so we report the two scales separately. At evaluation time, repairing the adapter restores the mechanism but not a significant outcome gain: parsing 0->84, rescues 0->9, pass rate 53->62 (n.s.). We release a 98-line preflight check that catches every silent failure here. The observed tool-call rate is not a property of the model alone; it is a property of the model-interface stack that measures it.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。