arXiv:2506.12801cs.AI2025-06

对比三种AI模型解密逻辑谜题,强化学习表现最佳。

Mastering Da Vinci Code: A Comparative Study of Transformer, LLM, and PPO-based Agents

  • 用PPO+Transformer处理完整游戏历史,策略更优
  • PPO Agent胜率58.5%,远超LLM类模型
  • 揭示大模型在长时逻辑推理中的局限性

《达·芬奇密码》是一款需要逻辑推理与处理不完全信息的智力游戏,对人工智能提出挑战。本文评估了三种不同架构的智能体:基于Transformer的基线模型(历史上下文有限)、多个大型语言模型(包括Gemini、DeepSeek和GPT系列)通过结构化提示引导,以及采用PPO算法并使用Transformer编码器处理完整游戏历史的智能体。实验结果表明,基于PPO的智能体胜率高达58.5% ± 1.0%,显著优于各类LLM模型。分析显示,深度强化学习在复杂推理任务中通过自对弈学习隐含策略方面具有优势。同时,尽管提示工程精巧,当前大模型仍难以在长时间游戏中保持严格逻辑一致性与战略深度。本研究有助于理解人工智能在包含隐藏信息与多步推理的休闲游戏中的应用,为智能体设计提供比较视角。

原文摘要 · Abstract (English)

The Da Vinci Code, a game of logical deduction and imperfect information, presents unique challenges for artificial intelligence, demanding nuanced reasoning beyond simple pattern recognition. This paper investigates the efficacy of various AI paradigms in mastering this game. We develop and evaluate three distinct agent architectures: a Transformer-based baseline model with limited historical context, several Large Language Model (LLM) agents (including Gemini, DeepSeek, and GPT variants) guided by structured prompts, and an agent based on Proximal Policy Optimization (PPO) employing a Transformer encoder for comprehensive game history processing. Performance is benchmarked against the baseline, with the PPO-based agent demonstrating superior win rates ($58.5\% \pm 1.0\%$), significantly outperforming the LLM counterparts. Our analysis highlights the strengths of deep reinforcement learning in policy refinement for complex deductive tasks, particularly in learning implicit strategies from self-play. We also examine the capabilities and inherent limitations of current LLMs in maintaining strict logical consistency and strategic depth over extended gameplay, despite sophisticated prompting. This study contributes to the broader understanding of AI in recreational games involving hidden information and multi-step logical reasoning, offering insights into effective agent design and the comparative advantages of different AI approaches.

强化学习逻辑推理游戏AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。