让大模型主动探索环境，提升决策能力。
Clearing the Fog: Towards Installing and Refining Proactive Exploration Capabilities in LLM Agents
- 构建丰富探索轨迹，缓解经验回溯偏差
- 对比学习区分有效探索与无效漫游
- 适合研究智能体自主决策的开发者
我们研究大模型智能体的主动探索能力，即通过探索环境获取信息以优化未来决策。为此，首先识别出两大核心瓶颈，并提出 ous，一种新型方法以实现并优化主动探索。该方法包含两个部分：(1) 探索数据构建，通过合成高探索性轨迹来缓解标准示范中的事后偏见；(2) 带对比信号引导的强化学习优化，利用对比轨迹对区分有效探索与无效徘徊。大量实验验证了 ous的有效性，并揭示了主动探索的特征。代码已公开：https://github.com/GuanZhizhao/SAFARI。
原文摘要 · Abstract (English)
We study proactive exploration in LLM agents, i.e., the ability to explore an environment to acquire information that improves future decision-making. In this regard, we first identify two fundamental bottlenecks that hinder this capability and then propose \ours, a novel method designed to instill and refine proactive exploration. Specifically, \ours\ consists of two components: (1) Exploratory Data Construction, which synthesizes exploration-rich trajectories to mitigate the hindsight bias of standard demonstrations; and (2) RL Optimization with Contrastive Signal Guidance, which leverages contrastive trajectory pairs to distinguish productive exploration from redundant wandering. Extensive experiments demonstrate the effectiveness of \ours\ and provide insights into the characteristics of proactive exploration. Our code is available at: https://github.com/GuanZhizhao/SAFARI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。