用DINO特征实现主体驱动的视频编辑，保持动作连贯性。
DIVE: Taming DINO for Subject-Driven Video Editing
- 用DINOv2提取语义特征作为隐式对应关系引导编辑。
- 在真实视频上实现高质量编辑，动作连贯性显著提升。
- 适合需要精准主体替换与运动一致性的视频编辑场景。
基于扩散模型在图像生成与编辑中的成功，视频编辑近期受到广泛关注。然而，保持时间一致性与动作对齐仍具挑战。为此，本文提出DINO引导的视频编辑框架（DIVE），支持基于目标文本提示或参考图像中特定身份的主体驱动编辑。DIVE的核心在于利用预训练DINOv2模型提取的强大语义特征作为隐式对应关系，指导编辑过程。为确保时间运动一致性，DIVE使用DINO特征对齐源视频的运动轨迹；为实现精确主体编辑，将参考图像的DINO特征注入预训练文生图模型，学习低秩适配（LoRAs），有效注册目标主体身份。在多样真实视频上的大量实验表明，该框架能实现高质量编辑结果，且具备强动作一致性，凸显了DINO在视频编辑中的潜力。
原文摘要 · Abstract (English)
Building on the success of diffusion models in image generation and editing, video editing has recently gained substantial attention. However, maintaining temporal consistency and motion alignment still remains challenging. To address these issues, this paper proposes DINO-guided Video Editing (DIVE), a framework designed to facilitate subject-driven editing in source videos conditioned on either target text prompts or reference images with specific identities. The core of DIVE lies in leveraging the powerful semantic features extracted from a pretrained DINOv2 model as implicit correspondences to guide the editing process. Specifically, to ensure temporal motion consistency, DIVE employs DINO features to align with the motion trajectory of the source video. For precise subject editing, DIVE incorporates the DINO features of reference images into a pretrained text-to-image model to learn Low-Rank Adaptations (LoRAs), effectively registering the target subject's identity. Extensive experiments on diverse real-world videos demonstrate that our framework can achieve high-quality editing results with robust motion consistency, highlighting the potential of DINO to contribute to video editing. Project page: https://dino-video-editing.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。