arXiv:2603.09283cs.CV2026-03被引 3

解决视频去物中阴影、抖动和遮罩缺陷问题,提升真实场景下的稳定性。

From Ideal to Real: Stable Video Object Removal under Imperfect Conditions

  • 采用窗口化遮罩合并策略,应对突发运动,避免遗漏目标区域。
  • 在合成数据上训练后,实测在缺陷遮罩下仍保持高精度与无闪烁。
  • 适合需要高鲁棒性的视频编辑、影视后期等实际应用场景。

视频中去除物体在存在阴影、突发动态和缺陷遮罩等现实干扰时仍具挑战性。现有基于扩散模型的视频修复方法常难以维持时间稳定性和视觉一致性。本文提出稳健视频物体移除框架SVOR,通过三项关键设计实现无阴影、无闪烁、容忍遮罩缺陷的移除:(1) 遮罩联合稳定擦除(MUSE),在时间遮罩下采样时采用窗口化并集策略,保留每个窗口内观测到的所有目标区域,有效应对突发动态,减少遗漏;(2) 去噪感知分割(DA-Seg),在解耦分支上部署轻量级分割头,结合去噪感知自适应归一化(AdaLN)并使用遮罩退化训练,提供内部扩散感知定位先验,不干扰内容生成;(3) 课程式两阶段训练:第一阶段在无配对的真实背景视频上进行自监督预训练,通过在线随机遮罩学习真实背景与时间先验;第二阶段在合成数据对上微调,引入遮罩退化与侧效权重损失,同时移除物体及其伴随的阴影/反光,增强跨域鲁棒性。大量实验表明,SVOR在多个数据集与退化遮罩基准上均达到新最佳性能,推动视频物体移除从理想设定迈向真实应用。

原文摘要 · Abstract (English)

Removing objects from videos remains difficult in the presence of real-world imperfections such as shadows, abrupt motion, and defective masks. Existing diffusion-based video inpainting models often struggle to maintain temporal stability and visual consistency under these challenges. We propose Stable Video Object Removal (SVOR), a robust framework that achieves shadow-free, flicker-free, and mask-defect-tolerant removal through three key designs: (1) Mask Union for Stable Erasure (MUSE), a windowed union strategy applied during temporal mask downsampling to preserve all target regions observed within each window, effectively handling abrupt motion and reducing missed removals; (2) Denoising-Aware Segmentation (DA-Seg), a lightweight segmentation head on a decoupled side branch equipped with Denoising-Aware AdaLN and trained with mask degradation to provide an internal diffusion-aware localization prior without affecting content generation; and (3) Curriculum Two-Stage Training: where Stage I performs self-supervised pretraining on unpaired real-background videos with online random masks to learn realistic background and temporal priors, and Stage II refines on synthetic pairs using mask degradation and side-effect-weighted losses, jointly removing objects and their associated shadows/reflections while improving cross-domain robustness. Extensive experiments show that SVOR attains new state-of-the-art results across multiple datasets and degraded-mask benchmarks, advancing video object removal from ideal settings toward real-world applications. Project page: https://xiaomi-research.github.io/svor/.

视频去物扩散模型鲁棒性真实场景

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。