arXiv:2506.04651cs.AI2025-06被引 21

让大模型在复杂游戏中自主进化策略,提升长期规划能力。

Agents of Change: Self-Evolving LLM Agents for Strategic Planning

  • 分离环境探索与策略优化,用可执行代码持续改进玩家行为
  • 在《卡坦岛》游戏中实现54%胜率,超越人类设计的最强基线
  • 适合研究长期决策、自进化系统或智能体协同的开发者

我们针对大语言模型智能体在长周期任务中的表现短板,提出一种持续学习的多智能体系统HexMachina,使其能在对抗性、随机性环境中保持一致的战略。以《卡坦岛》为挑战基准,成功需在随机性、交易、扩张与阻断中平衡短长期目标。传统提示驱动的智能体(如ReAct、Reflexion)每回合需重新解析复杂游戏状态,迅速填满上下文窗口并丧失战略连贯性。HexMachina通过分离环境发现(无文档诱导适配层)与策略改进(通过代码重构与仿真演化玩家),保留可执行产物,使大模型专注高层策略而非逐轮推理。在受控的Catanatron实验中,HexMachina从零开始学习,演化出胜率54%的玩家,超越最强人工基线AlphaBeta,并优于提示驱动和无环境发现的基线。消融实验表明,纯策略学习能显著提升性能。总体而言,以产物为中心的持续学习将大模型从脆弱的逐步决策者转变为稳定的策略设计者,推动长周期自主性的进展。

原文摘要 · Abstract (English)

We address the long-horizon gap in large language model (LLM) agents by enabling them to sustain coherent strategies in adversarial, stochastic environments. Settlers of Catan provides a challenging benchmark: success depends on balancing short- and long-term goals amid randomness, trading, expansion, and blocking. Prompt-centric LLM agents (e.g., ReAct, Reflexion) must re-interpret large, evolving game states each turn, quickly saturating context windows and losing strategic consistency. We propose HexMachina, a continual learning multi-agent system that separates environment discovery (inducing an adapter layer without documentation) from strategy improvement (evolving a compiled player through code refinement and simulation). This design preserves executable artifacts, allowing the LLM to focus on high-level strategy rather than per-turn reasoning. In controlled Catanatron experiments, HexMachina learns from scratch and evolves players that outperform the strongest human-crafted baseline (AlphaBeta), achieving a 54% win rate and surpassing prompt-driven and no-discovery baselines. Ablations confirm that isolating pure strategy learning improves performance. Overall, artifact-centric continual learning transforms LLMs from brittle stepwise deciders into stable strategy designers, advancing long-horizon autonomy.

大模型智能体策略规划持续学习游戏AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。