arXiv:2505.08459cs.AI2025-05中稿 · IJCNN 2025被引 3

让大模型学会识破对手策略并针对性反击,提升博弈表现。

Strategy-Augmented Planning for Large Language Models via Opponent Exploitation

  • 分两阶段构建策略评估网络,动态识别对手策略
  • 在MicroRTS中比基线提升85.35%,接近强化学习水平
  • 适合需要智能对抗的AI系统开发者参考

在对抗性领域高效建模与利用对手是长期挑战。基于海量文本训练的大语言模型(LLMs)在通用任务中表现优异,为对手建模带来新方向。现有方法多依赖提示词描述对手来生成决策,但仅适用于具备充分领域知识的场景。为此,我们提出两阶段策略增强规划(SAP)框架,通过引入策略评估网络(SEN)显著提升基于LLM代理的对手利用能力。离线阶段构建显式策略空间,并收集策略-结果对数据用于训练SEN;在线阶段,SAP动态识别对手策略,通过在已训练的SEN上搜索最优回应策略,再经精心设计的提示将策略转化为具体行动。实验表明,SAP具备强泛化能力,不仅对已见策略有效,也能应对未见过的新策略。在MicroRTS环境中,SAP相较基线方法性能提升85.35%,并与先进规则型AI具有相当竞争力。代码已开源。

原文摘要 · Abstract (English)

Efficiently modeling and exploiting opponents is a long-standing challenge in adversarial domains. Large Language Models (LLMs) trained on extensive textual data have recently demonstrated outstanding performance in general tasks, introducing new research directions for opponent modeling. Some studies primarily focus on directly using LLMs to generate decisions based on the elaborate prompt context that incorporates opponent descriptions, while these approaches are limited to scenarios where LLMs possess adequate domain expertise. To address that, we introduce a two-stage Strategy-Augmented Planning (SAP) framework that significantly enhances the opponent exploitation capabilities of LLM-based agents by utilizing a critical component, the Strategy Evaluation Network (SEN). Specifically, in the offline stage, we construct an explicit strategy space and subsequently collect strategy-outcome pair data for training the SEN network. During the online phase, SAP dynamically recognizes the opponent's strategies and greedily exploits them by searching best response strategy on the well-trained SEN, finally translating strategy to a course of actions by carefully designed prompts. Experimental results show that SAP exhibits robust generalization capabilities, allowing it to perform effectively not only against previously encountered opponent strategies but also against novel, unseen strategies. In the MicroRTS environment, SAP achieves a $85.35\%$ performance improvement over baseline methods and matches the competitiveness of reinforcement learning approaches against state-of-the-art (SOTA) rule-based AI. Our code is available at https://github.com/hsushuai/SAP.

博弈AI大模型应用策略规划

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。