提出新型代理置信度校准方法,提升AI自主系统可靠性
Agentic Confidence Calibration
- 构建全流程诊断框架HTC,捕捉代理执行轨迹中的动态特征
- 在8个基准上优于主流方法,出域测试中误差率最低
- 支持跨领域迁移与可解释性分析,适合高风险场景应用
AI代理正从被动语言模型演变为能执行复杂多步任务的自主系统。然而,其在失败时的过度自信仍是高风险场景部署的根本障碍。现有校准方法针对静态单轮输出设计,无法应对代理系统特有的挑战,如轨迹中错误累积、外部工具引入的不确定性及难以察觉的失败模式。为此,我们首次提出代理置信度校准问题,并提出全轨迹校准(HTC)框架,通过提取从宏观动态到微观稳定性的丰富过程级特征,实现对代理整个执行轨迹的诊断。该框架采用简单可解释的模型,在八个基准、多种大模型和不同代理架构下,持续超越强基线,在校准性和区分能力上表现优异。除性能外,HTC还带来三项关键进展:提供可解释性以揭示失败信号,支持跨领域迁移无需重训练,通过通用代理校准器(GAC)实现跨域泛化,在出域的GAIA基准上达到最低期望校准误差(ECE)。这些贡献确立了以过程为中心的新校准范式,为诊断和提升AI代理可靠性提供了有效框架。
原文摘要 · Abstract (English)
AI agents are rapidly advancing from passive language models to autonomous systems executing complex, multi-step tasks. Yet their overconfidence in failure remains a fundamental barrier to deployment in high-stakes settings. Existing calibration methods, built for static single-turn outputs, cannot address the unique challenges of agentic systems, such as compounding errors along trajectories, uncertainty from external tools, and opaque failure modes. To address these challenges, we introduce, for the first time, the problem of Agentic Confidence Calibration and propose Holistic Trajectory Calibration (HTC), a novel diagnostic framework that extracts rich process-level features ranging from macro dynamics to micro stability across an agent's entire trajectory. Powered by a simple, interpretable model, HTC consistently surpasses strong baselines in both calibration and discrimination, across eight benchmarks, multiple LLMs, and diverse agent frameworks. Beyond performance, HTC delivers three essential advances: it provides interpretability by revealing the signals behind failure, enables transferability by applying across domains without retraining, and achieves generalization through a General Agent Calibrator (GAC) that achieves the best calibration (lowest ECE) on the out-of-domain GAIA benchmark. Together, these contributions establish a new process-centric paradigm for confidence calibration, providing a framework for diagnosing and enhancing the reliability of AI agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。