测试大模型玩游戏能力，发现其随机性差、策略依赖提示设计。
Playing games with Large language models: Randomness and strategy
- 用程序化代理让大模型玩剪刀石头布和囚徒困境。
- 模型输出偏倚严重，重复对局后出现趋同或合作/竞争切换。
- 适合研究多智能体系统与模型决策局限性的读者。
博弈论长期以来被用于以简化形式描述复杂互动。本文探讨大语言模型（LLMs）是否能玩游戏，重点考察其在同时与顺序博弈中的随机化及策略适应能力。我们以 GPT-4o-Mini-2024-08-17 为对象，测试了两场模型间博弈：剪刀石头布（RPS）与囚徒困境（PD）。尽管常被描述为“随机鹦鹉”，但实验显示其输出在被要求随机时仍存在显著偏倚。研究发现，模型在重复博弈中表现出损失厌恶策略，RPS 对局趋于僵局，而 PD 则根据提示设计系统性地在合作与竞争间切换。文中详述了独立智能体交互的程序化工具及实现中遇到的代理型人工智能挑战。结果表明，大模型虽可玩游戏，但表现不佳。这对多智能体大模型系统的应用具有启示意义，并揭示了当前模型输出在战略决策中的局限性。
原文摘要 · Abstract (English)
Playing games has a long history of describing intricate interactions in simplified forms. In this paper we explore if large language models (LLMs) can play games, investigating their capabilities for randomisation and strategic adaptation through both simultaneous and sequential game interactions. We focus on GPT-4o-Mini-2024-08-17 and test two games between LLMs: Rock Paper Scissors (RPS) and games of strategy (Prisoners Dilemma PD). LLMs are often described as stochastic parrots, and while they may indeed be parrots, our results suggest that they are not very stochastic in the sense that their outputs - when prompted to be random - are often very biased. Our research reveals that LLMs appear to develop loss aversion strategies in repeated games, with RPS converging to stalemate conditions while PD shows systematic shifts between cooperative and competitive outcomes based on prompt design. We detail programmatic tools for independent agent interactions and the Agentic AI challenges faced in implementation. We show that LLMs can indeed play games, just not very well. These results have implications for the use of LLMs in multi-agent LLM systems and showcase limitations in current approaches to model output for strategic decision-making.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。