让小模型先推理，不确定时才调用大模型，降低成本同时保持高准确率。
ReDAct: Uncertainty-Aware Deferral for LLM Agents
- 小模型默认决策，不确定性高时转给大模型处理。
- 仅15%的决策需大模型参与，效果接近全程使用大模型。
- 适合对成本敏感且需高可靠性的智能体应用。
基于大语言模型（LLM）的智能体在复杂序列决策任务中日益流行，但其易产生幻觉，导致错误决策。在序列环境中，单次错误可能不可逆地破坏整个执行轨迹。尽管更大的模型幻觉更少，但每令牌成本显著更高。本文提出ReDAct（Reason-Defer-Act），采用双模型架构：一个小型廉价模型默认运行，一个大型可靠但昂贵的模型仅在小型模型预测不确定性超过校准阈值时启用。我们在ALFWorld和MiniGrid等文本式具身环境中评估该方法，结果表明，仅将约15%的决策委派给大模型即可达到与全程使用大模型相当的效果，同时大幅降低推理成本。
原文摘要 · Abstract (English)
Recently, LLM-based agents have become increasingly popular across many applications, including complex sequential decision-making problems. However, they inherit the tendency of LLMs to hallucinate, leading to incorrect decisions. In sequential settings, even a single mistake can irreversibly degrade the trajectory, making hallucinations an even bigger problem. Although larger LLMs hallucinate less, they incur a significantly higher per-token cost. In this paper, we address this tradeoff by proposing ReDAct (Reason-Defer-Act). In ReDAct, an agent is equipped with two LLMs: a small, cheap model used by default, and a large, more reliable but expensive model. When the predictive uncertainty of the small model exceeds a calibrated threshold, the decision is deferred to the large model. We evaluate our approach in text-based embodied environments such as ALFWorld and MiniGrid and show that deferring only about 15% of decisions to the large model can match the quality of using it exclusively, while significantly reducing inference costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。