动态选择关键帧作锚点，提升长视频生成连贯性。
DySink: Dynamic Frame Sinks for Autoregressive Long Video Generation
- 用检索机制动态选历史帧作锚点，替代固定早期帧
- 在50-100秒视频上超越现有自回归基线的时序质量
- 引入异常门过滤注意力过一致的上下文，防锚点坍塌
自回归长视频生成通常采用有界内存流式处理，结合局部窗口保证短期连续性与静态早期帧锚点实现长程依赖。但固定分配导致早期帧即使视觉状态已显著偏离仍被缓存，而可能更相关的中间历史被丢弃。这使得保留的长程上下文缺乏适应性，误导生成偏向过时线索；严重时，RoPE引起的相位重对齐会令跨头注意力趋同，引发锚点坍塌，内容退化至锚点帧。我们提出DySink，一种基于检索的框架，维护紧凑记忆库，并动态选取视觉相关的历史帧作为动态帧锚点。DySink将自适应检索与锚点异常门结合，过滤呈现过度跨头共识的检索上下文——这种模式与锚点坍塌相关。在50–100秒视频上的实验表明，DySink在评估的自回归基线中达到最高时序质量，同时保持良好的文本对齐与帧级质量。代码已公开于https://github.com/yebo0216best/DySink。
原文摘要 · Abstract (English)
Autoregressive long video generation often adopts bounded-memory streaming for efficiency, typically combining local windows for short-term continuity with static early-frame sinks as long-range anchors. However, this fixed allocation keeps early frames cached even when the current visual state has substantially diverged from them, while discarding potentially more relevant intermediate history. As a result, the retained long-range context may become less adaptive and bias generation toward outdated cues; in severe cases, RoPE-induced phase re-alignment can homogenize inter-head attention and cause sink collapse, where content regresses toward sink frames. We propose DySink, a retrieval-based framework that maintains a compact memory bank and selects visually relevant historical frames as dynamic frame sinks. DySink couples adaptive retrieval with a sink anomaly gate that filters retrieved context exhibiting excessive inter-head consensus, an attention pattern associated with sink collapse. Experiments on 50--100-second videos show that DySink achieves the highest measured temporal quality among the evaluated autoregressive baselines, while retaining competitive text alignment and framewise quality. The code is available at https://github.com/yebo0216best/DySink.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。