提出分阶段诊断框架，精准定位智能体调用失败原因。
SAAG: Structured Agent Assessment and Grounding
- 将调用评估拆解为注册合规、结构完整、参数锚定三阶段
- 在5~15个智能体的基准上，参数精度提升，幻觉减少
- 适合需要可解释性与自我修复能力的Agent开发场景
现有评测方式以精确匹配判断智能体调用是否成功，但会掩盖不同故障模式：模型可能选对函数却虚构参数值，或满足接口规范但基于错误原因调用。现有基准将其简化为二元得分，使开发者无法定位问题。本文提出SAAG——一种分层诊断框架，将调用评估分解为注册合规、结构完整性和参数锚定三个顺序阶段，每个阶段输出可解释的诊断信号。这些信号支持迭代自修复：预测失败时，阶段特定反馈引导针对性修正，不泄露真实值。在基于Glaive函数调用数据集构建的可控基准上，使用三个参数量小于40亿的本地模型，测试了5、10、15个智能体规模。结构化反馈显著提升参数精度，减少值幻觉，而端到端F1提升有限且依赖模型。结果表明，分阶段诊断是理解并改进跨模型族与注册规模下调用可靠性的必要视角。
原文摘要 · Abstract (English)
Exact-match evaluation of agent-calling obscures qualitatively different failure modes: a model may select the right function yet hallucinate argument values, or satisfy a schema while choosing a agent for the wrong reason. Existing benchmarks collapse these distinctions into a single binary score, leaving practitioners unable to diagnose where agent calls fail. We propose SAAG a cascaded diagnostic framework that decomposes agent-calling evaluation into three sequential stages: registry conformance, structural completeness, and argument grounding, each producing interpretable stage-specific diagnostics. These diagnostics additionally enable iterative self-repair: on prediction failure, the stage-specific signal guides targeted correction without leaking ground-truth values. We evaluate this framework on a controlled benchmark derived from Glaive's function-calling dataset across registry sizes of 5, 10, and 15 agents using three local sub-4B-parameter models. Structured feedback consistently improves argument precision and reduces value hallucination relative to single-pass inference and uninformative binary feedback, while end-to-end F1 gains are modest and model-dependent. These results suggest that stage-decomposed diagnostic evaluation is a necessary lens for understanding and improving agent-calling reliability across model families and registry scales.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。