arXiv:2410.07901cs.CV2024-10ECCV被引 19

用半监督方法提升真实雪景视频去雪效果,解决合成数据泛化差问题。

Semi-Supervised Video Desnowing Network via Temporal Decoupling Experts and Distribution-Driven Contrastive Regularization

  • 基于分布对比正则化缓解合成与真实数据分布差异
  • 在85段真实雪景视频上实现更优的去雪效果
  • 适合需要真实场景泛化能力的视频去雪研究者

雪害在户外场景中造成严重视觉退化,阻碍计算机视觉任务发展。现有深度学习去雪方法在合成数据集上表现良好,但在真实雪景视频上因缺乏成对训练数据而性能下降。为此,我们构建了一个包含85段真实雪景视频的全新数据集,并提出半监督视频去雪网络SemiVDN,引入分布驱动的对比正则化,有效缩小合成与真实数据之间的分布差距,保持雪无关背景细节。此外,基于大气散射模型,设计先验引导的时间解耦专家模块,以帧相关方式分解雪景视频的物理成分。在基准数据集和自建真实数据上评估表明,该方法优于当前最先进的图像与视频级去雪方法。

原文摘要 · Abstract (English)

Snow degradations present formidable challenges to the advancement of computer vision tasks by the undesirable corruption in outdoor scenarios. While current deep learning-based desnowing approaches achieve success on synthetic benchmark datasets, they struggle to restore out-of-distribution real-world snowy videos due to the deficiency of paired real-world training data. To address this bottleneck, we devise a new paradigm for video desnowing in a semi-supervised spirit to involve unlabeled real data for the generalizable snow removal. Specifically, we construct a real-world dataset with 85 snowy videos, and then present a Semi-supervised Video Desnowing Network (SemiVDN) equipped by a novel Distribution-driven Contrastive Regularization. The elaborated contrastive regularization mitigates the distribution gap between the synthetic and real data, and consequently maintains the desired snow-invariant background details. Furthermore, based on the atmospheric scattering model, we introduce a Prior-guided Temporal Decoupling Experts module to decompose the physical components that make up a snowy video in a frame-correlated manner. We evaluate our SemiVDN on benchmark datasets and the collected real snowy data. The experimental results demonstrate the superiority of our approach against state-of-the-art image- and video-level desnowing methods.

视频去雪半监督分布对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。