测试大模型在反事实博弈中的真实策略推理能力。
Evaluating Counterfactual Strategic Reasoning in Large Language Models
- 设计反事实版囚徒困境和剪刀石头布游戏
- 发现模型对奖励变化不敏感，泛化能力差
- 适合研究模型是否真会策略思考的学者
我们在重复博弈场景中评估大语言模型（LLMs）的表现，以判断其策略行为是源于真实推理，还是依赖记忆模式。针对经典的囚徒困境（PD）和剪刀石头布（RPS）游戏，我们引入反事实变体，改变收益结构与动作标签，打破原有对称性和占优关系。通过多指标评估框架，对比默认版本与反事实版本的表现，揭示了模型在激励敏感性、结构泛化性和反事实环境下的策略推理能力存在明显局限。
原文摘要 · Abstract (English)
We evaluate Large Language Models (LLMs) in repeated game-theoretic settings to assess whether strategic performance reflects genuine reasoning or reliance on memorized patterns. We consider two canonical games, Prisoner's Dilemma (PD) and Rock-Paper-Scissors (RPS), upon which we introduce counterfactual variants that alter payoff structures and action labels, breaking familiar symmetries and dominance relations. Our multi-metric evaluation framework compares default and counterfactual instantiations, showcasing LLM limitations in incentive sensitivity, structural generalization and strategic reasoning within counterfactual environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。