用求解器生成策略,让大模型学会复杂博弈中的均衡玩法。
Solver-Guided Reasoning for Mixed-Equilibrium Strategies

- 用求解器输出构建可解释的混合策略树,替代人类数据。
- 在德州扑克中使模型策略与均衡的差距减少52.6%。
- 适合想提升博弈推理能力的研究者或游戏AI开发者。
大语言模型的推理通常依赖人类文本、示范和理由,但在复杂博弈的均衡推理中,人类数据常因直觉和启发式而偏离真实均衡。尤其在混合策略均衡中,人类行为严重偏向纯策略,导致模型训练效果差。为此,本文提出混合策略决策树(MDT),将求解器输出的均衡最优性转化为人类与模型都能理解的稀疏规则。利用求解器而非人工标注,可扩展至任意新状态和延续场景。我们在无限制德州扑克上通过查询求解器获得超过2.5亿个混合策略决策,结合其他技术使8种不同LLM配置的ℓ₁距离均衡值降低52.6%。路由消融实验验证了基于影子对比的增量贡献,完整河牌局与说谎者骰子实验则评估了策略保真度和跨场景迁移能力。
原文摘要 · Abstract (English)
Reasoning in large language models (LLMs) is often grounded in human text, human demonstrations, and human-generated rationales. For equilibrium reasoning in complex games, however, relying on human data can be suboptimal. In fact, human play is often guided by intuition and heuristics and can deviate substantially from game equilibrium. This discrepancy is amplified in games with mixed-strategy equilibria, where human data is heavily biased toward pure strategies. Consequently, conditioning LLMs on this data yields weak game strategies. To grant LLMs the reasoning capacity in games, in this work, we study how to elicit equilibrium play using solver output. We propose Mixed-Strategy Decision Tree (MDT), which articulates the silent optimality of the equilibrium into sparse strategic rules that both humans and LLMs could understand. Using solver output rather than human annotation allows us to extend the input to arbitrarily new states and continuations. We instantiate this study on No-Limit Texas Hold'em by querying a solver oracle for over \textbf{250 million mixed-strategy decisions}; MDT together with other techniques \textbf{reduces the $\ell_1$ distance to the equilibrium by $52.6\%$} across $8$ different LLM configurations. A Route-only ablation tests the incremental contribution of the shadow-based contrast, while complete River-endgame and Liar's Dice experiments evaluate strategic fidelity and portability beyond the original NLH communication setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。