让视频生成模型持续响应不断到来的编辑指令,实现无限流式编辑。
InfinityEdit: Infinite Video Editing with a Lightweight Edit-Ignition Adapter

- 设计轻量级编辑适配器,仅在编辑指令到达时激活,其余时间保持原模型生成能力。
- 在连续编辑序列中保持画面连贯性,生成质量不随编辑次数下降。
- 适用于直播重制、实时镜头运动等开放流式视频编辑场景。
现有基于大模型的视频编辑方法多依赖固定时长输入帧的逐帧修改,无法处理开放流式场景(如直播游戏重风格化或持续镜头移动)。本文提出无限视频编辑任务:给定前序片段和编辑请求,模型需持续生成后续内容并实时应用编辑,且该过程可无限延续。为此,我们构建了用于无限视频编辑的数据收集流程,并提出InfinityEdit——一个轻量级编辑适配器。其包含三个注意力模块:历史交叉注意力利用输入帧引导去噪,时间因果自注意力确保时序信息单向流动,编辑交叉注意力注入编辑指令。推理时,仅在编辑指令到达的片段激活适配器,后续片段由原始模型以重置锚点帧生成。该方案在不破坏原模型无限生成能力的前提下完成编辑,实验表明其能忠实延续视频流,并在无限制编辑序列中保持稳定。
原文摘要 · Abstract (English)
With large pretrained models, existing methods have effectively improved instruction-based video editing. However, most of them rely on an in-place editing assumption. They align the edited video with the given source clip frame by frame over a fixed time span. This pattern fails for open-ended streams, e.g., restyling a live game or applying a camera move to an ongoing shot. In such cases, edits must extend to future frames as they arrive, rather than be applied to a static input clip. In this paper, we study this setting and name it infinite video editing: given a preceding segment and an edit request, a model must generate the next segment that continues the stream while applying the requested edit. This process repeats as an unbounded sequence of edit instructions arrives. This task brings two challenges: the edit must be a faithful continuation rather than a frame-wise rewrite, and generation quality must remain stable as edits accumulate. To address them, we first design a data-collection pipeline for infinite video editing. Based on the collected data, we propose InfinityEdit, a lightweight edit adapter that equips a streaming video generator with unbounded editing ability. The adapter contains three attention modules. History cross-attention guides the denoising frames using the input frames. Temporal causal self-attention keeps temporal cues flowing only from earlier frames to later ones. Edit cross-attention injects the edit request into generation. During inference, the adapter is activated only in the chunk where an edit request arrives. Subsequent chunks are generated by the original model with a reset anchor frame. This scheme applies the edit while preserving the original model's infinite generation ability. Extensive experiments show that InfinityEdit faithfully continues the stream under each edit, and stays stable over unbounded edit sequences.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。