arXiv:2502.01181cs.CV2025-02ICCV被引 27

无需标注损坏区域,模型自动识别并修复视频缺损。

BVINet: Unlocking Blind Video Inpainting with Zero Annotations

  • 自动生成损坏区域掩码,同时完成内容修复。
  • 在真实与合成数据集上实现领先修复效果。
  • 适合无标注视频修复场景,如监控、老片修复。

视频修复旨在填补视频中受损区域的合理内容。现有方法通常假设损坏位置已知,主要关注“如何修复”,需人工绘制二值掩码以指示修复位置,但标注成本高,限制了实用性。本文提出一种新的盲视频修复设定,使网络能直接从受损视频学习到修复结果,无需损坏区域标注。我们设计端到端的盲视频修复网络BVINet,同时解决“何处修复”和“如何修复”问题:一方面,通过检测帧内语义不连续区域并利用视频时序一致性先验来预测掩码;另一方面,将预测掩码融入网络,引导模型从完整区域获取上下文信息进行修复。此外,引入一致性损失正则化训练参数,使掩码预测与修复过程相互约束,提升整体性能。我们构建了一个包含合成与真实损坏视频及其修复结果的数据集,为该领域研究提供宝贵资源。大量实验表明,本方法在多种场景下均显著优于现有方法。

原文摘要 · Abstract (English)

Video inpainting aims to fill in corrupted regions of the video with plausible contents. Existing methods generally assume that the locations of corrupted regions are known, focusing primarily on the "how to inpaint". This reliance necessitates manual annotation of the corrupted regions using binary masks to indicate "whereto inpaint". However, the annotation of these masks is labor-intensive and expensive, limiting the practicality of current methods. In this paper, we expect to relax this assumption by defining a new blind video inpainting setting, enabling the networks to learn the mapping from corrupted video to inpainted result directly, eliminating the need of corrupted region annotations. Specifically, we propose an end-to-end blind video inpainting network (BVINet) to address both "where to inpaint" and "how to inpaint" simultaneously. On the one hand, BVINet can predict the masks of corrupted regions by detecting semantic-discontinuous regions of the frame and utilizing temporal consistency prior of the video. On the other hand, the predicted masks are incorporated into the BVINet, allowing it to capture valid context information from uncorrupted regions to fill in corrupted ones. Besides, we introduce a consistency loss to regularize the training parameters of BVINet. In this way, mask prediction and video completion mutually constrain each other, thereby maximizing the overall performance of the trained model. Furthermore, we customize a dataset consisting of synthetic corrupted videos, real-world corrupted videos, and their corresponding completed videos. This dataset serves as a valuable resource for advancing blind video inpainting research. Extensive experimental results demonstrate the effectiveness and superiority of our method.

视频修复盲修复自监督时序建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。