测试大模型何时该随机决策，发现强模型能根据对手调整策略。
Do LLMs Know When to Flip a Coin? Strategic Randomization through Reasoning and Experience
- 设计类田忌赛马的零和博弈，用纳什均衡验证随机化能力。
- 强模型在提示下随机性显著提升，弱模型始终确定性决策。
- 适合研究抽象推理与自适应学习的大模型开发者参考。
战略随机化是博弈论中的核心原则，但在大语言模型（LLMs）中仍缺乏深入探索。以往研究常混淆随机决策与机械生成随机数，导致评估不完整。为此，我们提出一种受田忌赛马启发的新型零和博弈，其纳什均衡对应最大熵策略。该游戏复杂性使未经训练的人类和低水平模型难以察觉此性质。我们在五种提示风格（明确、中性、暗示）下，通过系统提供的随机选择进行多轮对抗比赛，隔离出模型对随机化的决策能力。结果显示，弱模型无论提示如何均保持确定性；强模型在明确提示下随机性显著增强。面对弱模型时，强模型采用确定性策略以利用其偏差；而面对同级对手时，则趋向均衡策略。通过胜负结果与贝叶斯因子分析，我们证明了不同模型在战略推理能力上的显著差异，揭示了提升抽象推理与自适应学习的潜在方向。代码已开源，确保可复现性。
原文摘要 · Abstract (English)
Strategic randomization is a key principle in game theory, yet it remains underexplored in large language models (LLMs). Prior work often conflates the cognitive decision to randomize with the mechanical generation of randomness, leading to incomplete evaluations. To address this, we propose a novel zero-sum game inspired by the Tian Ji Horse Race, where the Nash equilibrium corresponds to a maximal entropy strategy. The game's complexity masks this property from untrained humans and underdeveloped LLMs. We evaluate five LLMs across prompt styles -- framed, neutral, and hinted -- using competitive multi-tournament gameplay with system-provided random choices, isolating the decision to randomize. Results show that weaker models remain deterministic regardless of prompts, while stronger models exhibit increased randomization under explicit hints. When facing weaker models, strong LLMs adopt deterministic strategies to exploit biases, but converge toward equilibrium play when facing peers. Through win/loss outcomes and Bayes factor analysis, we demonstrate meaningful variation in LLMs' strategic reasoning capabilities, highlighting opportunities for improvement in abstract reasoning and adaptive learning. We make our implementation publicly available at https://github.com/ocelopus/llm-when-to-throw-coin to ensure full reproducibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。