为复杂交互式LLM智能体建立可信的不确定性评估框架。
Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunities
- 提出首个通用的智能体不确定性评估框架，涵盖多种现有方法。
- 在真实智能体基准τ²-bench上发现四类关键技术挑战。
- 面向安全应用，为未来研究指明方向，适合关注AI安全的读者。
大语言模型（LLM）的不确定性量化（UQ）是保障日常应用安全的关键。然而，尽管LLM智能体日益用于复杂任务，多数UQ研究仍局限于单轮问答场景。本文主张UQ研究需转向交互式智能体的真实环境，并提出构建智能体UQ的三大支柱：(1) 基础：首次提出通用的智能体UQ形式化，涵盖广泛现有设置；(2) 挑战：识别四大与智能体场景相关的难题——不确定性估计器选择、异构实体的不确定性、交互系统中不确定性动态建模、缺乏细粒度基准，并在真实世界基准τ²-bench上进行数值分析；(3) 未来方向：讨论智能体UQ的实际影响与未解问题，为后续探索提供前瞻性思考。
原文摘要 · Abstract (English)
Uncertainty quantification (UQ) for large language models (LLMs) is a key building block for safety guardrails of daily LLM applications. Yet, even as LLM agents are increasingly deployed in highly complex tasks, most UQ research still centers on single-turn question-answering. We argue that UQ research must shift to realistic settings with interactive agents, and that a new principled framework for agent UQ is needed. This paper presents three pillars to build a solid ground for future agent UQ research: (1. Foundations) We present the first general formulation of agent UQ that subsumes broad classes of existing UQ setups; (2. Challenges) We identify four technical challenges specifically tied to agentic setups -- selection of uncertainty estimator, uncertainty of heterogeneous entities, modeling uncertainty dynamics in interactive systems, and lack of fine-grained benchmarks -- with numerical analysis on a real-world agent benchmark, $τ^2$-bench; (3. Future Directions) We conclude with noting on the practical implications of agent UQ and remaining open problems as forward-looking discussion for future explorations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。