用简单方法实现高质量视频编辑，支持多种输入指令。
EasyV2V: A High-quality Instruction-based Video Editing Framework
- 用单帧监督和伪配对构建多样视频数据集
- 轻量LoRA微调+序列拼接，效果超越现有系统
- 支持文本、掩码、参考图等灵活控制方式
尽管图像编辑发展迅速，视频编辑仍面临一致性、控制力和泛化性挑战。本文研究数据、架构与控制的设计空间，提出EasyV2V——一种基于指令的高效视频编辑框架。数据层面，通过具快速逆变换的现成专家组合生成多样化视频对，利用单帧监督将图像编辑对扩展为视频，构建共享仿射运动的伪配对，并挖掘密集标注片段获取视频对，同时加入过渡监督以学习编辑过程演化。模型层面，发现预训练文生视频模型具备编辑能力，因此采用简化设计：仅通过简单序列拼接进行条件引导，配合轻量级LoRA微调即可训练出强模型。控制方面，统一使用单个掩码机制实现时空控制，并支持可选参考图像。整体上，EasyV2V可处理灵活输入（如视频+文本、视频+掩码+文本、视频+掩码+参考+文本），在多项指标上超越同期及商用系统，达到当前最佳水平。
原文摘要 · Abstract (English)
While image editing has advanced rapidly, video editing remains less explored, facing challenges in consistency, control, and generalization. We study the design space of data, architecture, and control, and introduce \emph{EasyV2V}, a simple and effective framework for instruction-based video editing. On the data side, we compose existing experts with fast inverses to build diverse video pairs, lift image edit pairs into videos via single-frame supervision and pseudo pairs with shared affine motion, mine dense-captioned clips for video pairs, and add transition supervision to teach how edits unfold. On the model side, we observe that pretrained text-to-video models possess editing capability, motivating a simplified design. Simple sequence concatenation for conditioning with light LoRA fine-tuning suffices to train a strong model. For control, we unify spatiotemporal control via a single mask mechanism and support optional reference images. Overall, EasyV2V works with flexible inputs, e.g., video+text, video+mask+text, video+mask+reference+text, and achieves state-of-the-art video editing results, surpassing concurrent and commercial systems. Project page: https://snap-research.github.io/easyv2v/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。