不训练即可提升大模型决策多样性，解决重复动作问题。
DORA Explorer: Improving the Exploration Ability of LLMs Without Training
- 通过序列级概率统计评分生成候选动作，实现推理时多样性优化
- 在文本冒险环境中的成功率从31.43%提升至45.5%（Qwen-2.5 7B）
- 无需训练，适用于多种模型，适合强化学习与决策类任务
大型语言模型代理在序列决策中难以生成多样化输出，导致探索不足、解次优及重复行为。现有采样策略（如温度调节）仅在词元层面引入多样性，而非序列层面。本文提出DORA EXPLORER（面向多样性的动作排序），一种无需训练的推理阶段算法。该方法生成多个候选动作，基于序列级对数概率统计进行评分，并通过可调探索参数采样。在经典多臂赌博机设置中，DORA显著优于温度采样。主评估在文本冒险学习环境套件TALES上进行，提示策略无法有效探索，而DORA在不同模型族中均带来稳定提升，例如Qwen-2.5 7B在TextWorld中成功率从31.43%提升至45.5%。此外，DORA还能避免陷入循环等常见失败。项目地址：https://dora-explore.github.io/
原文摘要 · Abstract (English)
Large language model (LLM) agents for sequential decision-making struggle to produce diverse outputs. This leads to insufficient exploration, suboptimal solutions, and repeated actions. Actions are generated at the sequence level, but existing sampling strategies, such as temperature scaling, introduce diversity at the token level, not at the sequence level. We introduce DORA EXPLORER (Diversity-Oriented Ranking of Actions), a training-free, inference-time algorithm for improving exploration in LLM agents. DORA generates multiple candidate actions, scores them using sequence-level log-probability statistics, and samples an action via a tunable exploration parameter. We first study exploration in the classic Multi-Armed Bandit setting, where DORA substantially outperforms temperature-based sampling. Our main evaluation is on the Text Adventure Learning Environment Suite (TALES), where prompting strategies fail to explore but DORA delivers consistent gains across model families, e.g., 31.43% (ReAct) -> 45.5% (DORA) for Qwen-2.5 7B in TextWorld. Beyond exploration, DORA prevents common failures, such as getting stuck in loops. Our project is available at: https://dora-explore.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。