arXiv:2506.02477cs.CV2025-06被引 2

通过模拟大脑记忆机制,让去雨模型持续学习新雨天数据。

Towards Better De-raining Generalization via Rainy Characteristics Memorization and Replay

  • 用GAN提取新数据特征,类比海马体记忆
  • 结合真实与生成数据训练,提升泛化能力
  • 适合需要长期适应复杂雨天场景的开发者

当前图像去雨方法主要依赖有限数据集,导致在真实多变雨天条件下表现不佳。为此,我们提出一种新框架,使网络能通过不断接入新增数据集逐步扩展去雨知识库,显著提升适应性。受人类大脑持续吸收与泛化经验能力启发,该方法借鉴互补学习系统机制:首先利用生成对抗网络(GAN)捕捉并保留新数据的独特特征,模拟海马体的学习记忆功能;随后,去雨网络在已有数据与GAN合成数据上联合训练,类比海马体重播与交错学习过程;此外,采用知识蒸馏技术,利用重播数据复现海马体重播激活与原有皮层知识之间的协同作用。该综合框架使去雨网络能够从多个数据集积累知识,持续提升对未见雨天场景的性能。在三个基准去雨网络上的测试表明,该框架不仅实现了跨六个数据集的持续知识积累,还在泛化到新真实雨天场景方面超越现有最先进方法。

原文摘要 · Abstract (English)

Current image de-raining methods primarily learn from a limited dataset, leading to inadequate performance in varied real-world rainy conditions. To tackle this, we introduce a new framework that enables networks to progressively expand their de-raining knowledge base by tapping into a growing pool of datasets, significantly boosting their adaptability. Drawing inspiration from the human brain's ability to continuously absorb and generalize from ongoing experiences, our approach borrow the mechanism of the complementary learning system. Specifically, we first deploy Generative Adversarial Networks (GANs) to capture and retain the unique features of new data, mirroring the hippocampus's role in learning and memory. Then, the de-raining network is trained with both existing and GAN-synthesized data, mimicking the process of hippocampal replay and interleaved learning. Furthermore, we employ knowledge distillation with the replayed data to replicate the synergy between the neocortex's activity patterns triggered by hippocampal replays and the pre-existing neocortical knowledge. This comprehensive framework empowers the de-raining network to amass knowledge from various datasets, continually enhancing its performance on previously unseen rainy scenes. Our testing on three benchmark de-raining networks confirms the framework's effectiveness. It not only facilitates continuous knowledge accumulation across six datasets but also surpasses state-of-the-art methods in generalizing to new real-world scenarios.

去雨持续学习GAN知识蒸馏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。