无需成对数据,用无监督方法分离雨滴与背景,实现单图去雨滴。
Unsupervised Network for Single Image Raindrop Removal
- 基于循环网络架构,分离雨滴层、透明掩码和干净背景层。
- 在多个基准数据集上,定量指标与视觉效果均优于现有方法。
- 适合缺乏成对训练数据的现实场景,如监控与自动驾驶视觉系统。
雨滴导致的图像质量下降是影响视觉系统性能的重要难题。现有大多数去雨滴算法依赖成对图像的监督学习,但在真实场景中难以获取。本文提出一种基于无监督学习的深度神经网络,仅需含雨滴与无雨滴的两组未配对图像即可训练。模型采用循环网络结构进行层级分离,将含雨图像分解为雨滴层、透明掩码和干净背景层。其中,干净背景层即为目标去雨结果,透明掩码指示雨滴空间位置。此外,模型引入反馈机制,利用高层信息迭代优化低层表示:前一轮输出作为下一轮输入,结合原始含雨图像,逐步去除雨滴。大量实验表明,该方法在多个雨滴基准数据集上均取得优异的定量指标与视觉效果。
原文摘要 · Abstract (English)
Image quality degradation caused by raindrops is one of the most important but challenging problems that reduce the performance of vision systems. Most existing raindrop removal algorithms are based on a supervised learning method using pairwise images, which are hard to obtain in real-world applications. This study proposes a deep neural network for raindrop removal based on unsupervised learning, which only requires two unpaired image sets with and without raindrops. Our proposed model performs layer separation based on cycle network architecture, which aims to separate a rainy image into a raindrop layer, a transparency mask, and a clean background layer. The clean background layer is the target raindrop removal result, while the transparency mask indicates the spatial locations of the raindrops. In addition, the proposed model applies a feedback mechanism to benefit layer separation by refining low-level representation with high-level information. i.e., the output of the previous iteration is used as input for the next iteration, together with the input image with raindrops. As a result, raindrops could be gradually removed through this feedback manner. Extensive experiments on raindrop benchmark datasets demonstrate the effectiveness of the proposed method on quantitative metrics and visual quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。