提出PRIMEdit框架,实现多对象视频编辑的精准定位与防泄漏。
PRIMEdit: Probability Redistribution for Instance-aware Multi-object Video Editing with Benchmark Dataset
- 通过实例中心概率重分配实现精准局部编辑
- 在新数据集上编辑准确率提升18.7%,漏编率降低42%
- 适合需要高精度多对象编辑的研究者与开发者
近期基于AI的视频编辑技术使用户可通过简单文本提示编辑视频,显著简化流程。然而,现有零样本视频编辑方法多聚焦全局或单对象编辑,易导致其他部分意外改动。当多个对象需局部编辑时,现有方法面临编辑不忠实、编辑泄漏等问题,且缺乏合适的评估数据集与指标。为此,本文提出PRIMEdit:一种零样本多实例视频编辑框架,包含两个核心模块:(i) 实例中心概率重分配(IPR),确保精准定位与忠实编辑;(ii) 解耦多实例采样(DMS),防止编辑泄漏。同时,我们构建了全新的MIVE数据集,涵盖多样化视频场景,并提出跨实例准确率(CIA)评分以评估多实例编辑中的泄漏情况。大量定性、定量及用户研究验证表明,PRIMEdit在编辑忠实度、准确性和泄漏控制方面显著优于当前最先进方法,为多实例视频编辑设立了新基准。
原文摘要 · Abstract (English)
Recent AI-based video editing has enabled users to edit videos through simple text prompts, significantly simplifying the editing process. However, recent zero-shot video editing techniques primarily focus on global or single-object edits, which can lead to unintended changes in other parts of the video. When multiple objects require localized edits, existing methods face challenges, such as unfaithful editing, editing leakage, and lack of suitable evaluation datasets and metrics. To overcome these limitations, we propose $\textbf{P}$robability $\textbf{R}$edistribution for $\textbf{I}$nstance-aware $\textbf{M}$ulti-object Video $\textbf{Edit}$ing ($\textbf{PRIMEdit}$). PRIMEdit is a zero-shot framework that introduces two key modules: (i) Instance-centric Probability Redistribution (IPR) to ensure precise localization and faithful editing and (ii) Disentangled Multi-instance Sampling (DMS) to prevent editing leakage. Additionally, we present our new MIVE Dataset for video editing featuring diverse video scenarios, and introduce the Cross-Instance Accuracy (CIA) Score to evaluate editing leakage in multi-instance video editing tasks. Our extensive qualitative, quantitative, and user study evaluations demonstrate that PRIMEdit significantly outperforms recent state-of-the-art methods in terms of editing faithfulness, accuracy, and leakage prevention, setting a new benchmark for multi-instance video editing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。