arXiv:2605.16143cs.AIcs.CL2026-05被引 3

让大模型先探路再行动,提升复杂环境适应力

Look Before You Leap: Autonomous Exploration for LLM Agents

论文配图:Look Before You Leap: Autonomous Exploration for LLM Agents
图 1 · 摘自论文原文
  • 用探索检查点覆盖率量化智能体发现关键信息的能力
  • 新训练策略使智能体探索更全面,下游任务性能显著提升
  • 适合需要自主感知与决策的现实场景应用

基于大语言模型的智能体常因过早执行而失败,源于在未充分了解环境前就依赖已有知识。本文提出自主探索是构建自适应智能体的关键能力。为此,我们引入可验证的「探索检查点覆盖率」指标,衡量智能体对关键状态、物体和交互可能性的发现广度。系统评估显示,传统任务导向强化学习训练的智能体行为狭窄且重复,影响后续表现。为此,我们设计一种训练策略:交替进行任务执行与探索滚动生成,分别使用对应的可验证奖励进行优化。基于此,提出「先探索后执行」范式:智能体首先利用交互预算获取环境实证知识,再用于任务求解。结果表明,系统性学习探索对构建可泛化、真实可用的智能体至关重要。

原文摘要 · Abstract (English)

Large language model based agents often fail in unfamiliar environments due to premature exploitation: a tendency to act on prior knowledge before acquiring sufficient environment-specific information. We identify autonomous exploration as a critical yet underexplored capability for building adaptive agents. To formalize and quantify this capability, we introduce Exploration Checkpoint Coverage, a verifiable metric that measures how broadly an agent discovers key states, objects, and affordances. Our systematic evaluation reveals that agents trained with standard task-oriented reinforcement learning consistently exhibit narrow and repetitive behaviors that impede downstream performance. To address this limitation, we develop a training strategy that interleaves task-execution rollouts and exploration rollouts, with each type of rollout optimized by its corresponding verifiable reward. Building on this training strategy, we propose the Explore-then-Act paradigm, which decouples information-gathering from task execution: agents first utilize an interaction budget to acquire grounded environmental knowledge, then leverage it for task resolution. Our results demonstrate that learning to systematically explore is imperative for building generalizable and real-world-ready agents.

智能体自主探索强化学习大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。