arXiv:2608.28460cs.CV2026-08

让视频生成模型记住过去,精准召回历史信息以保持长时一致性。

LayerRecall: A State-Conditioned Memory Router for Long-Horizon Consistency in Video Generation

论文配图:LayerRecall: A State-Conditioned Memory Router for Long-Horizon Consistency in Video Generation
图 1 · 摘自论文原文
  • 根据当前状态选择性地从记忆中召回历史键值对,仅注入特定层级的骨干网络。
  • 在100个多镜头评测中,比现有方法更有效恢复长期出现的物体与场景。
  • 无需大量高质量长视频数据,可跨模型复用且推理开销极低。

自回归视频扩散通过有限近期上下文生成视频块实现可扩展的长视频生成。尽管基于时效性的缓存能维持局部连续性,但会丢弃主题、对象、场景或属性再次出现所需的早期线索。现有记忆机制虽引入非局部历史,但访问不等于有效使用。分析发现,视频DiT层对当前、近期和遥远上下文有不同偏好,表明长程记忆需同时决定‘取什么’和‘在哪用’。我们提出LayerRecall:一种当前条件化的层选择性记忆路由机制,仅将相关的历史键值状态注入特定敏感层,其余部分保留本地注意力。为减少对稀缺高质量长视频和显式记忆标签的依赖,进一步提出跨时序预测匹配(CHPM),利用具备长上下文的参考模型在预测空间监督受限记忆路由。在100个多镜头评估提示下,LayerRecall在MemoBench和MovieBench上取得最佳综合表现,并在VBench-Long上达到其骨干模型性能,展示更强的长程恢复能力而不牺牲局部连续性。定性分析显示,记忆引导的自我修正机制使初始错配的局部属性能在不重置运动或场景结构的前提下回归历史外观。附加分析表明其具备跨骨干迁移性及可忽略的推理开销。

原文摘要 · Abstract (English)

Autoregressive video diffusion enables scalable long-video generation by producing chunks from a bounded recent context. While recency-based caching preserves local continuity, it evicts historical cues needed when subjects, objects, scenes, or attributes reappear. Existing memory mechanisms expose models to nonlocal history, but access alone does not ensure effective use. Our analysis reveals that video DiT layers exhibit distinct preferences for current, recent, and distant context, suggesting that long-range memory requires deciding both what to retrieve and where to use it. We introduce LayerRecall, a current-conditioned, layer-selective memory router that retrieves relevant historical K/V states and injects them only into backbone-specific memory-sensitive layers while preserving local attention elsewhere. To reduce reliance on scarce high-quality long-horizon videos and explicit memory-allocation labels, we further propose Cross-Horizon Prediction Matching (CHPM), which uses a privileged long-context reference to supervise the bounded-memory router in prediction space. Across 100 multi-shot evaluation prompts, LayerRecall achieves the best overall results on MemoBench and MovieBench while matching its backbone on VBench-Long, demonstrating stronger long-range recovery without sacrificing local continuity. Qualitative analyses further reveal memory-guided self-correction, whereby initially mismatched local attributes return to their historical appearance without resetting ongoing motion or scene structure. Additional analyses show cross-backbone portability and negligible inference overhead.

视频生成长时一致性记忆路由

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。