用文本世界模型提升星际争霸智能体决策能力，实现30%胜率提升。
World Models for Policy Refinement in StarCraft II
- 构建可条件生成的文本世界模型，模拟未来观察状态。
- 通过生成-模拟-优化循环，使智能体在困难难度下胜率提升30%。
- 方法与历史摘要技术互补，适合强化学习与游戏智能研究者。
大型语言模型（LLMs）展现出强大的推理能力，推动其在复杂决策环境中的应用。星际争霸II（SC2）因其庞大的状态-动作空间和部分可观测性，成为极具挑战性的测试平台。然而，现有基于LLM的SC2智能体主要聚焦于改进策略本身，尚未探索将可学习、动作条件化的动态模型整合进决策循环。本文提出StarWM与StarWM-Agent，并首次系统研究玩家视角、动作条件化文本世界模型在SC2中的可学习性与决策价值。StarWM在部分可观测条件下预测短期未来观测。StarWM-Agent将StarWM集成至轻量级生成-模拟-精炼循环中，实现前瞻驱动的策略优化。大量实验表明，StarWM显著优于零样本基线，而StarWM-Agent在硬（LV5）、更难（LV6）、极难（LV7）难度下分别获得30%、15%、30%的胜率提升。额外分析显示，该方法与现有历史摘要型智能体方法互补，二者结合在在线评估中表现最强。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have recently shown strong reasoning capabilities, motivating their use in complex decision-making environments. StarCraft II (SC2), with its massive state-action space and partial observability, is a challenging testbed. However, existing LLM-based SC2 agents primarily focus on improving the policy itself, leaving the integration of a learnable, action-conditioned dynamics model into the decision loop largely unexplored. In this work, we propose StarWM and StarWM-Agent, and conduct the first systematic study of the learnability and decision utility of player-view, action-conditioned textual world models for SC2. StarWM predicts short-horizon future observations under partial observability. StarWM-Agent integrates StarWM into a lightweight Generate-Simulate-Refine loop for foresight-driven policy refinement. Extensive experiments show that StarWM substantially outperforms zero-shot baselines across multiple dimensions, while StarWM-Agent achieves consistent win-rate gains of 30%, 15%, and 30% against the SC2 built-in AI at Hard (LV5), Harder (LV6), and VeryHard (LV7), respectively. Additional analyses show that our method is complementary to existing history-summarization SC2-agent approaches, with their combination achieving the strongest online performance in our evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。