用时间序列数据生成雨痕原型,高效实现单图去雨
A Prototype Unit for Image De-raining using Time-Lapse Data
- 设计雨痕原型单元,从时序数据中提取实时雨痕特征
- 在多个基准上达到顶尖性能,优于当前主流方法
- 适合需要轻量化部署的去雨场景,如移动端应用
针对单图去雨任务,即从带雨图像中恢复无雨背景,本文提出一种新方法:雨痕原型单元(RsPU)。该单元利用真实世界时序数据,高效编码与雨痕相关的特征为实时原型,避免了传统方法对大量内存资源的依赖。所提去雨网络结合编码器-解码器结构与RsPU,通过注意力机制学习并封装多样化的雨痕相关特征。为保证效果,引入包含凝聚与分散项的特征原型损失函数,以捕捉原型特征的紧凑性与多样性。实验在多个去雨基准上验证了方法的有效性,并进行了详尽消融分析,结果表明其在各类雨景图像中均能取得与当前最优方法相当甚至更优的表现。
原文摘要 · Abstract (English)
We address the challenge of single-image de-raining, a task that involves recovering rain-free background information from a single rain image. While recent advancements have utilized real-world time-lapse data for training, enabling the estimation of consistent backgrounds and realistic rain streaks, these methods often suffer from computational and memory consumption, limiting their applicability in real-world scenarios. In this paper, we introduce a novel solution: the Rain Streak Prototype Unit (RsPU). The RsPU efficiently encodes rain streak-relevant features as real-time prototypes derived from time-lapse data, eliminating the need for excessive memory resources. Our de-raining network combines encoder-decoder networks with the RsPU, allowing us to learn and encapsulate diverse rain streak-relevant features as concise prototypes, employing an attention-based approach. To ensure the effectiveness of our approach, we propose a feature prototype loss encompassing cohesion and divergence components. This loss function captures both the compactness and diversity aspects of the prototypical rain streak features within the RsPU. Our method evaluates various de-raining benchmarks, accompanied by comprehensive ablation studies. We show that it can achieve competitive results in various rain images compared to state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。