无需训练，让世界模型自动生成用户指定的视觉元素。
From Zero to Hero: Training-Free Custom Concept Spawning in World Models
- 用锚点替换法在生成过程中注入新概念
- 概念保持光照、尺度和视角一致，时间连贯
- 支持图像或文字输入，适合游戏与互动叙事
自回归世界模型已成为交互式视频生成的强大范式，允许用户通过动作在动态生成的环境中导航。这些模型通常基于文本提示和/或单个参考帧生成整个世界。然而，当用户移动到该帧不可见区域时，未见区域由模型基础先验填充，缺乏用户指定内容和位置的机制。这在游戏、互动叙事和仿真等需可控场景构图的应用中构成根本性限制。我们称此缺失能力为概念生成（concept spawning）——即在世界模型中引入用户指定的视觉概念，类似于游戏引擎中的物体生成。本文提出SPAWN（Swapping Pinned Anchor with Windowed iNjection），一种无需训练的概念生成方法。SPAWN利用图像到视频骨干网络的结构特性：上下文记忆的第一个槽位固定于参考帧，作为每段生成的基准锚点。通过在短注入窗口内将该锚点替换为外部概念的潜在表示，并让原锚点返回，即可使概念通过模型自身记忆自然传播至后续生成中。SPAWN支持从角色、道具等细粒度实体到建筑、地标等大规模元素，输入可为概念图像或文本描述。实验表明，SPAWN能实现概念在光照、尺度、视角上的一致性，同时保持身份和时间连贯性，证明在现有自回归世界模型中无需训练即可实现可控概念生成。
原文摘要 · Abstract (English)
Autoregressive world models have emerged as a powerful paradigm for interactive video generation, allowing users to navigate dynamically generated environments through actions. These models are typically conditioned on a text prompt and/or a single reference frame, from which the entire world is generated. Yet the moment the user navigates beyond what is visible in that frame, the unseen regions are populated by the base model's priors, with no mechanism for the user to specify what should appear and where. This is a fundamental limitation for applications such as gaming, interactive storytelling, and simulation, where controllable scene composition is essential. We refer to this missing capability as concept spawning; introducing a user-specified visual concept into a world model, analogous to spawning in a game engine. We introduce SPAWN (Swapping Pinned Anchor with Windowed iNjection), a training-free method for concept spawning. SPAWN exploits a structural property of image-to-video backbones: the first slot of the context memory is pinned to the reference frame and acts as a foundational anchor for every generated chunk. By swapping this anchor with an external concept latent over a short injection window and letting the original anchor return, we cause the concept to propagate naturally through the rollout via the model's own memory. SPAWN supports concepts from fine-grained entities such as characters and props to large-scale elements such as buildings and landmarks, and accepts either a concept image or a text description as input. Experiments show that SPAWN integrates concepts with consistent lighting, scale, and perspective while preserving identity and temporal coherence, demonstrating that controllable concept spawning is achievable in existing autoregressive world models without any training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。