让视频生成模型记住长期场景，又不拖慢速度。
WorldKV: Efficient World Memory with World Retrieval and Compression
- 用检索与压缩机制保存长期视觉记忆，不重编码
- 在固定内存下存储两倍历史，保持画面一致性
- 无需训练，适合实时生成与长时交互场景
自回归视频扩散模型实现了实时、动作驱动的世界生成。然而，维持持久世界——即回看先前视角仍能获得一致内容——仍是开放问题。全量KV缓存虽能保证一致性，但内存和注意力开销随生成长度线性增长，破坏实时性；滑动窗口推理虽恢复吞吐，却丢弃长期一致性。本文提出WorldKV，一种免训练框架，包含两个组件：世界检索（World Retrieval）将被踢出的KV缓存块存入GPU/CPU内存，并通过相机/动作对应关系，选择性检索场景相关块，原样插入注意力窗口，无需重新编码；世界压缩（World Compression）通过关键帧间相似性，剪枝每块内冗余令牌，使单块存储减半，在固定预算下可容纳两倍历史。在Matrix-Game-2.0与LingBot-World-Fast数据集上，WorldKV在约2倍吞吐下达到或超越全KV缓存的一致性，且无需微调即媲美训练过的基线。
原文摘要 · Abstract (English)
Autoregressive video diffusion models have enabled real-time, action-conditioned world generation. However, sustaining a persistent world, where revisiting a previously seen viewpoint yields consistent content, remains an open problem. Full KV-cache attention preserves this consistency but breaks real-time constraints: memory footprint and attention cost grow linearly with rollout length. Sliding window inference restores throughput but discards long-term consistency. We propose WorldKV, a training-free framework with two components: World Retrieval and World Compression. World Retrieval stores evicted KV-cache chunks in GPU/CPU memory and selectively retrieves scene-relevant chunks via camera/ action correspondence, inserting them back into the native attention window without re-encoding. World Compression prunes redundant tokens within each chunk via key-key similarity to an anchor frame, halving per-chunk storage to fit 2x more history under a fixed budget. On Matrix-Game-2.0 and LingBot- World-Fast, WorldKV matches or exceeds full-KV memory fidelity at roughly 2x the throughput, and is competitive with memory-trained baselines without any fine-tuning. Project Page: https://cvlab-kaist.github.io/WorldKV/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。