用线性探测发现大模型工具调用错误，效果显著且可泛化。
The Calls are Coming from Inside the Model: Investigating Probe-based Detection of Tool-Calling Errors in LLMs
- 通过线性探针分析模型隐藏状态，识别工具调用错误。
- 在18个模型上验证，能检测值错但类型对的隐性错误。
- 适合关注模型安全与调试的研究者和开发者。
大型语言模型（LLMs）的隐藏状态包含丰富的知识与行为信息，这些信息难以仅通过输入输出观察获得。随着基于LLM的系统越来越多地与外部世界交互，如何检测工具调用中的错误成为关键问题。为此，我们研究了线性探针在检测错误工具调用方面的有效性，在伯克利函数调用排行榜上的18个工具调用型LLM上进行了评估。结果表明，探针能有效识别多种工具调用错误，包括参数值错误但类型正确的情形——这类错误通常无法被标准日志框架记录。模型规模、探测层位置及模型微调类型是影响探测效果的重要因素。此外，探针还展现出对新型错误的泛化能力，这对实际部署至关重要。
原文摘要 · Abstract (English)
The hidden states of large language models (LLMs) are known to capture rich information relating to model knowledge and behavior that can be hard to extract from examination of input and output alone. As LLM-based systems increasingly interface with the external world, one area of concern is detecting incorrect or improper use of tools. Motivated by this, we study the effectiveness of using linear probes to detect incorrect tool-calls, measuring probe efficacy across 18 tool-calling LLMs evaluated on the Berkeley Function Calling Leaderboard. Overall, we find that probing is an effective means to catch a range of different tool-calling errors, including errors arising from using an argument that has the wrong value but the correct type, which might not be recorded by standard logging frameworks. Important factors in success include model size, probing layer, and model post-training type. We also show that probes are capable of generalizing to novel types of errors, which is critical in real world deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。