无需掩码和参考图，用文字指令实现视频物体的精准删加
LoVoRA: Text-guided and Mask-free Video Object Removal and Addition with Learnable Object-aware Localization
- 通过可学习的物体感知定位机制，实现时空一致的编辑控制
- 利用扩散掩码预测器，端到端完成物体删除与添加任务
- 适用于需要高自由度编辑的视频创作场景
文本引导的视频编辑，尤其是物体移除与添加，因需精确的空间与时序一致性而极具挑战。现有方法通常依赖辅助掩码或参考图像进行编辑引导，限制了其可扩展性和泛化能力。为此，我们提出 LoVoRA，一种基于物体感知定位机制的无掩码视频物体移除与添加框架。该方法采用独特的数据构建流程，融合图像到视频转换、基于光流的掩码传播与视频修复技术，实现时序一致的编辑效果。核心创新在于可学习的物体感知定位机制，为物体插入与移除任务提供密集的时空监督。通过扩散掩码预测器，LoVoRA 实现端到端视频编辑，推理阶段无需外部控制信号。大量实验与人工评估验证了其有效性和高质量表现。
原文摘要 · Abstract (English)
Text-guided video editing, particularly for object removal and addition, remains a challenging task due to the need for precise spatial and temporal consistency. Existing methods often rely on auxiliary masks or reference images for editing guidance, which limits their scalability and generalization. To address these issues, we propose LoVoRA, a novel framework for mask-free video object removal and addition using object-aware localization mechanism. Our approach utilizes a unique dataset construction pipeline that integrates image-to-video translation, optical flow-based mask propagation, and video inpainting, enabling temporally consistent edits. The core innovation of LoVoRA is its learnable object-aware localization mechanism, which provides dense spatio-temporal supervision for both object insertion and removal tasks. By leveraging a Diffusion Mask Predictor, LoVoRA achieves end-to-end video editing without requiring external control signals during inference. Extensive experiments and human evaluation demonstrate the effectiveness and high-quality performance of LoVoRA. https://cz-5f.github.io/LoVoRA.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。