让敌方AI模仿玩家策略，逼其不断改变打法。
Mirror Mode in Fire Emblem: Beating Players at their own Game with Imitation and Reinforcement Learning
- 用生成对抗模仿学习+行为克隆+近端策略优化混合模型模仿玩家动作
- 玩家测试显示防御策略模仿效果好，进攻策略仍有差距
- 玩家认可自身战术被复现，整体满意度提升，适合想挑战自我的玩家
回合制游戏中敌方策略应具有不可预测性。本文提出镜像模式（Mirror Mode），让敌方AI模仿玩家个人策略，迫使玩家持续调整玩法。在Unity中搭建了简化版《火焰之纹章：英雄》游戏，包含标准模式与镜像模式。第一阶段实验确定了合适的模仿模型：结合生成对抗模仿学习（GAIL）、行为克隆（BC）和近端策略优化（PPO）。第二阶段通过玩家实验评估模型，模型基于参与者提供的操作示范进行训练。结果显示，模型对防守行为有较好模仿，但进攻策略仍不足。玩家问卷表明其能识别出自身撤退战术，镜像模式整体满意度更高。进一步优化模型可提升模仿质量，增强玩家体验，尤其当玩家面对自己策略时。
原文摘要 · Abstract (English)
Enemy strategies in turn-based games should be surprising and unpredictable. This study introduces Mirror Mode, a new game mode where the enemy AI mimics the personal strategy of a player to challenge them to keep changing their gameplay. A simplified version of the Nintendo strategy video game Fire Emblem Heroes has been built in Unity, with a Standard Mode and a Mirror Mode. Our first set of experiments find a suitable model for the task to imitate player demonstrations, using Reinforcement Learning and Imitation Learning: combining Generative Adversarial Imitation Learning, Behavioral Cloning, and Proximal Policy Optimization. The second set of experiments evaluates the constructed model with player tests, where models are trained on demonstrations provided by participants. The gameplay of the participants indicates good imitation in defensive behavior, but not in offensive strategies. Participant's surveys indicated that they recognized their own retreating tactics, and resulted in an overall higher player-satisfaction for Mirror Mode. Refining the model further may improve imitation quality and increase player's satisfaction, especially when players face their own strategies. The full code and survey results are stored at: https://github.com/YannaSmid/MirrorMode
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。