用半监督方法提升真实雪景视频去雪效果,解决合成数据泛化差问题。
Semi-Supervised Video Desnowing Network via Temporal Decoupling Experts and Distribution-Driven Contrastive Regularization
- 基于分布对比正则化缓解合成与真实数据分布差异
- 在85段真实雪景视频上实现更优的去雪效果
- 适合需要真实场景泛化能力的视频去雪研究者
雪害在户外场景中造成严重视觉退化,阻碍计算机视觉任务发展。现有深度学习去雪方法在合成数据集上表现良好,但在真实雪景视频上因缺乏成对训练数据而性能下降。为此,我们构建了一个包含85段真实雪景视频的全新数据集,并提出半监督视频去雪网络SemiVDN,引入分布驱动的对比正则化,有效缩小合成与真实数据之间的分布差距,保持雪无关背景细节。此外,基于大气散射模型,设计先验引导的时间解耦专家模块,以帧相关方式分解雪景视频的物理成分。在基准数据集和自建真实数据上评估表明,该方法优于当前最先进的图像与视频级去雪方法。
原文摘要 · Abstract (English)
Snow degradations present formidable challenges to the advancement of computer vision tasks by the undesirable corruption in outdoor scenarios. While current deep learning-based desnowing approaches achieve success on synthetic benchmark datasets, they struggle to restore out-of-distribution real-world snowy videos due to the deficiency of paired real-world training data. To address this bottleneck, we devise a new paradigm for video desnowing in a semi-supervised spirit to involve unlabeled real data for the generalizable snow removal. Specifically, we construct a real-world dataset with 85 snowy videos, and then present a Semi-supervised Video Desnowing Network (SemiVDN) equipped by a novel Distribution-driven Contrastive Regularization. The elaborated contrastive regularization mitigates the distribution gap between the synthetic and real data, and consequently maintains the desired snow-invariant background details. Furthermore, based on the atmospheric scattering model, we introduce a Prior-guided Temporal Decoupling Experts module to decompose the physical components that make up a snowy video in a frame-correlated manner. We evaluate our SemiVDN on benchmark datasets and the collected real snowy data. The experimental results demonstrate the superiority of our approach against state-of-the-art image- and video-level desnowing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。