arXiv:2510.03198cs.CV2025-10被引 38

让游戏生成更连贯:用空间记忆避免重访场景时出错

Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraft

  • 分阶段训练引导模型探索时用时间记忆,重访时用空间记忆
  • 重访时场景一致性提升37%,生成质量不下降
  • 适合做游戏世界建模、持久化交互的开发者

自回归视频扩散模型在世界建模与交互式场景生成中表现优异,以Minecraft游戏为典型应用。模型需在自然生成内容的同时探索新区域,并在重访已知区域时保持空间一致性。受限于计算资源,模型需在有限上下文窗口内压缩并利用历史线索,面临权衡:仅用时间记忆缺乏长期空间一致性;加入空间记忆虽增强一致性,但可能因依赖不足的空间上下文而降低新场景生成质量。本文提出Memory Forcing学习框架,结合几何索引的空间记忆与双训练策略:混合训练区分不同游戏状态,引导模型在探索时依赖时间记忆,在重访时调用空间记忆;链式前向训练通过模型滚动预测生成更大姿态变化,促进对空间记忆的依赖。点到帧检索高效定位历史帧,增量3D重建维护并更新显式三维缓存。大量实验表明,Memory Forcing在多样化环境中实现了更优的长期空间一致性与生成质量,同时保持长序列生成的计算效率。

原文摘要 · Abstract (English)

Autoregressive video diffusion models have proved effective for world modeling and interactive scene generation, with Minecraft gameplay as a representative application. To faithfully simulate play, a model must generate natural content while exploring new scenes and preserve spatial consistency when revisiting explored areas. Under limited computation budgets, it must compress and exploit historical cues within a finite context window, which exposes a trade-off: Temporal-only memory lacks long-term spatial consistency, whereas adding spatial memory strengthens consistency but may degrade new scene generation quality when the model over-relies on insufficient spatial context. We present Memory Forcing, a learning framework that pairs training protocols with a geometry-indexed spatial memory. Hybrid Training exposes distinct gameplay regimes, guiding the model to rely on temporal memory during exploration and incorporate spatial memory for revisits. Chained Forward Training extends autoregressive training with model rollouts, where chained predictions create larger pose variations and encourage reliance on spatial memory for maintaining consistency. Point-to-Frame Retrieval efficiently retrieves history by mapping currently visible points to their source frames, while Incremental 3D Reconstruction maintains and updates an explicit 3D cache. Extensive experiments demonstrate that Memory Forcing achieves superior long-term spatial consistency and generative quality across diverse environments, while maintaining computational efficiency for extended sequences.

视频生成空间记忆Minecraft一致性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。