提出内部置信度估计算法，让工具调用智能体更安全可靠。
MICE for CATs: Model-Internal Confidence Estimation for Calibrating Agents with Tools
- 通过解析语言模型各层输出，计算与最终结果的相似性来评估置信度。
- 在模拟试错数据集上，置信度校准误差低于基线，工具调用效用显著提升。
- 零样本泛化到未见API，适用于高风险场景下的安全决策。
使用工具的智能体需要既有效又安全。良好的置信度校准有助于权衡行动的风险与收益，但现有模型普遍校准不佳。受可解释性研究启发，本文提出一类新型模型内部置信度估计器（MICE），用于更准确评估调用工具时的置信度。MICE首先利用logitLens解码语言模型各中间层输出，再计算每层生成结果与最终输出之间的相似性得分，将这些特征输入学习的概率分类器以评估置信度。在基于Llama3模型的模拟试错（STE）工具调用数据集上，MICE在平滑期望校准误差（smoothed expected calibration error）上优于或匹配基线。使用MICE置信度决定是否调用工具，在新提出的期望工具调用效用（expected tool-calling utility）指标上显著超越强基线。进一步实验表明，MICE样本效率高，能零样本泛化至未见过的API，且在不同风险水平场景中均带来更高工具调用效用。代码已开源，地址：https://github.com/microsoft/mice_for_cats。
原文摘要 · Abstract (English)
Tool-using agents that act in the world need to be both useful and safe. Well-calibrated model confidences can be used to weigh the risk versus reward of potential actions, but prior work shows that many models are poorly calibrated. Inspired by interpretability literature exploring the internals of models, we propose a novel class of model-internal confidence estimators (MICE) to better assess confidence when calling tools. MICE first decodes from each intermediate layer of the language model using logitLens and then computes similarity scores between each layer's generation and the final output. These features are fed into a learned probabilistic classifier to assess confidence in the decoded output. On the simulated trial and error (STE) tool-calling dataset using Llama3 models, we find that MICE beats or matches the baselines on smoothed expected calibration error. Using MICE confidences to determine whether to call a tool significantly improves over strong baselines on a new metric, expected tool-calling utility. Further experiments show that MICE is sample-efficient, can generalize zero-shot to unseen APIs, and results in higher tool-calling utility in scenarios with varying risk levels. Our code is open source, available at https://github.com/microsoft/mice_for_cats.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。