arXiv:2605.24674cs.CV2026-05

让视频编辑模型学会分步推理,提升指令理解与细节保持能力

Reasoning to Align: Implicit Reasoning in Diffusion Transformers for Video Editing

论文配图:Reasoning to Align: Implicit Reasoning in Diffusion Transformers for Video Editing
图 1 · 摘自论文原文
  • 用可学习的编辑令牌分层注入,实现从粗到细的编辑流程
  • 通过参考分支最大化注意力特征互信息,约束内部推理过程
  • 在局部和复合编辑任务上显著优于现有方法,适合复杂视频修改

基于指令的视频编辑需根据自然语言指令转换源视频,同时保留无关内容并保持时间连贯性。现有扩散变换器(DiT)编辑器因两个结构缺陷表现不佳:其一,条件信号未区分地输入所有变压器模块,导致单一标记流同时编码全局编辑意图与细粒度视觉证据;其二,控制编辑的交叉注意力模式仅通过像素级重建间接监督,使模型内部推理过程缺乏约束。为解决上述问题,我们提出RVEDiT框架,包含两个互补组件:一是粒度路由标记条件机制,将来自多模态大模型的可学习编辑标记注入浅层模块,而深层模块保留原始视觉与文本标记,从而在主干网络中实现粗到细的编辑流程;二是参考锚定注意力对齐机制,在训练中引入参数共享的参考分支,并最大化编辑分支与参考分支注意力特征间的互信息,以正则化模型内部推理,且不增加推理开销。在标准指令式视频编辑基准上的实验表明,RVEDiT持续优于当前最优基线,尤其在局部及组合编辑任务上表现突出。

原文摘要 · Abstract (English)

Instruction-based video editing requires transforming a source video according to a natural-language instruction while preserving irrelevant content and remaining temporally coherent. We argue that existing Diffusion Transformer (DiT) editors struggle with this task for two structural reasons. First, conditioning signals are fed undifferentiated into all transformer blocks, forcing a single token stream to encode both global editing intent and fine-grained visual evidence. Second, the cross-attention patterns that govern the edit are supervised only indirectly through pixel-level reconstruction, leaving the model's internal reasoning process under-constrained. To address both limitations, we propose RVEDiT, an implicit Reasoning Video Editing DiT framework built around two complementary components. The first, Granularity-Routed Token Conditioning, introduces learnable editing tokens distilled from a multimodal LLM and routes them to shallow blocks, while reserving native visual and textual tokens for deeper blocks, thereby inducing a coarse-to-fine editing process inside the backbone. The second, Reference-Anchored Attention Alignment, employs a parameter-sharing reference branch during training and maximizes the mutual information between the attention features of the editing and reference branches, regularizing the model's internal reasoning without incurring any additional inference cost. Experiments on standard instruction-based video editing benchmarks show that RVEDiT consistently outperforms state-of-the-art baselines, with particularly strong gains on localized and compositional edits.

视频编辑扩散模型推理对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。