大模型能学会复杂纸牌游戏,还能同时掌握多个游戏规则。
Can Large Language Models Master Complex Card Games?
- 用高质量对战数据微调大模型,使其接近强人工智能水平。
- 同一模型可同时掌握多个纸牌游戏,相似规则间性能互补。
- 兼顾通用能力时需加入一定通用指令数据,避免能力退化。
复杂游戏长期是检验人工智能算法进展的重要基准。AlphaGo、AlphaZero 和 MuZero 在围棋和国际象棋中击败顶尖人类选手,引发社会广泛关注。与此同时,大语言模型(LLMs)在各类任务中展现出惊人能力,引发了关于其是否能在复杂游戏中取得类似成就的疑问。本文系统评估了大模型在八种不同纸牌游戏中的学习能力,考察了在高质量对战数据上微调的影响,以及模型在掌握这些游戏的同时保持通用能力的能力。结果表明:(1)通过在高质量数据上监督微调,大模型可接近强游戏AI的性能;(2)大模型能同时在多个复杂纸牌游戏中达到一定熟练度,规则相似的游戏间性能有增益,规则差异大的则存在冲突;(3)掌握复杂游戏会导致通用能力下降,但通过引入适量通用指令数据可有效缓解这一现象。评估结果展示了大模型强大的学习能力和泛化潜力。代码已开源:https://github.com/THUDM/LLM4CardGame
原文摘要 · Abstract (English)
Complex games have long been an important benchmark for testing the progress of artificial intelligence algorithms. AlphaGo, AlphaZero, and MuZero have defeated top human players in Go and Chess, garnering widespread societal attention towards artificial intelligence. Concurrently, large language models (LLMs) have exhibited remarkable capabilities across various tasks, raising the question of whether LLMs can achieve similar success in complex games. In this paper, we explore the potential of LLMs in mastering complex card games. We systematically assess the learning capabilities of LLMs across eight diverse card games, evaluating the impact of fine-tuning on high-quality gameplay data, and examining the models' ability to retain general capabilities while mastering these games. Our findings indicate that: (1) LLMs can approach the performance of strong game AIs through supervised fine-tuning on high-quality data, (2) LLMs can achieve a certain level of proficiency in multiple complex card games simultaneously, with performance augmentation for games with similar rules and conflicts for dissimilar ones, and (3) LLMs experience a decline in general capabilities when mastering complex games, but this decline can be mitigated by integrating a certain amount of general instruction data. The evaluation results demonstrate strong learning ability and versatility of LLMs. The code is available at https://github.com/THUDM/LLM4CardGame
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。