提出分层记忆压缩机制，让自回归视频生成更长时仍保持一致性。
FadeMem: Distance-Aware Memory Consolidation for Autoregressive Video Diffusion
- 按时间距离动态合并历史记忆块，近处细粒度、远处稀疏化
- 在固定缓存预算下生成多分钟视频，峰值内存更低
- 无需修改架构，提升长视频连贯性，适合长序列生成任务
自回归视频生成通过逐段生成长视频，但历史键值缓存随长度增长。现有有限缓存方法使用局部窗口、沉降标记或压缩状态，但通常为历史不同部分分配固定角色。我们提出FadeMem，一种距离感知的键值记忆整合机制，在固定缓存预算下将历史键值块组织成时间层级结构。该设计基于频率相关的时序衰减：细节快速去相关，而粗粒度场景结构和身份信息在更长时程中仍有效。生成时，新历史以细粒度条目插入，旧的相邻条目按幂律时间分配策略逐步合并，实现单缓存内密集近、稀疏远的记忆分布。无需架构改动，FadeMem在保持视觉质量的同时显著提升长程一致性，轻量级适配进一步增强运动动态与视觉保真度。在同一固定缓存预算下，该方法对多分钟至小时级视频生成均有效，并降低峰值内存占用。
原文摘要 · Abstract (English)
Autoregressive video generators synthesize long videos by generating successive temporal segments, but their historical KV cache grows with video length. Existing bounded-cache methods reduce this cost with local windows, sink tokens, or compressed memory states, yet they usually assign fixed roles to different parts of the history. We propose FadeMem, a distance-aware KV memory consolidation mechanism that organizes historical KV blocks into a temporal hierarchy under a fixed cache budget. This design is motivated by frequency-dependent temporal decay: fine details decorrelate quickly, while coarse scene structure and identity remain useful over longer horizons. During generation, new history is inserted as fine-grained entries, while older adjacent entries are progressively merged under a power-law temporal allocation schedule, yielding a dense-near, sparse-far memory within one cache. Without architectural changes, FadeMem improves long-range consistency while largely preserving visual quality, and lightweight adaptation further enhances motion dynamics and visual fidelity. Using the same unified schedule under a fixed cache budget, FadeMem also remains effective over multi-minute and hour long video generation and reduces peak memory under a matched KV budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。