arXiv:2501.08325cs.CV2025-01ICCV被引 127

用AI生成可交互的全新游戏视频,支持任意场景和动作控制。

GameFactory: Creating New Games with Generative Interactive Videos

论文配图:GameFactory: Creating New Games with Generative Interactive Videos
图 1 · 摘自论文原文
  • 构建动作可控的通用游戏视频生成框架,支持键盘鼠标精准控制。
  • 实现跨场景的动作通用性,可在不同风格游戏中保持一致控制效果。
  • 适合游戏开发、AI内容生成研究者,推动自动化游戏创作发展。

生成式视频有望彻底改变游戏开发方式,实现内容自动生成。本文提出GameFactory,一个支持动作控制且场景泛化的游戏视频生成框架。为解决动作可控性难题,我们构建了无偏见的GF-Minecraft动作标注数据集,并设计动作控制模块,实现对键盘与鼠标输入的精确控制。进一步支持自回归生成,实现无限长度的交互视频。更重要的是,GameFactory克服了现有方法普遍缺乏的场景泛化能力。为生成全新多样游戏,我们利用预训练视频扩散模型的开放域生成先验。为弥合开放域先验与小规模游戏数据集间的领域差距,提出多阶段训练策略与领域适配器,解耦游戏风格学习与动作控制,使动作控制不再依赖特定游戏风格,从而实现真正的场景泛化。实验表明,GameFactory能有效生成开放域、动作可控的游戏视频,标志着人工智能驱动游戏生成的重要进展。

原文摘要 · Abstract (English)

Generative videos have the potential to revolutionize game development by autonomously creating new content. In this paper, we present GameFactory, a framework for action-controlled scene-generalizable game video generation. We first address the fundamental challenge of action controllability by introducing GF-Minecraft, an action-annotated game video dataset without human bias, and developing an action control module that enables precise control over both keyboard and mouse inputs. We further extend to support autoregressive generation for unlimited-length interactive videos. More importantly, GameFactory tackles the critical challenge of scene-generalizable action control, which most existing methods fail to address. To enable the creation of entirely new and diverse games beyond fixed styles and scenes, we leverage the open-domain generative priors from pre-trained video diffusion models. To bridge the domain gap between open-domain priors and small-scale game datasets, we propose a multi-phase training strategy with a domain adapter that decouples game style learning from action control. This decoupling ensures that action control learning is no longer bound to specific game styles, thereby achieving scene-generalizable action control. Experimental results demonstrate that GameFactory effectively generates open-domain action-controllable game videos, representing a significant step forward in AI-driven game generation.

游戏生成视频生成动作控制扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。