用共享知识库优化卷积参数,实现持续图像修复无遗忘
Sharing the Learned Knowledge-base to Estimate Convolutional Filter Parameters for Continual Image Restoration
- 通过可复用的知识库动态调整卷积滤波器参数
- 新任务引入后旧任务性能不下降,且新任务效果随经验积累提升
- 无需修改主干结构,计算开销小,适合实际部署
持续学习是深度学习中模型持续学习新任务而不遗忘旧知识的新兴方向。该领域虽有诸多进展,但在图像修复任务中研究较少。大尺寸图像和多样的退化类型给修复任务带来独特挑战。现有方法需对网络结构进行大量改造,导致显著计算开销;正则化方法也不适用,因不同修复任务需不同特征处理方式。为此,我们提出一种简单的卷积层修改方案,可在不改动主干架构的前提下,复用先前任务的知识来适应新任务。该方法能增加可训练参数,却几乎不增加计算开销或推理时间。实验表明,新任务可被无缝引入而不会损害已有任务性能;同时,借助前序任务构建的知识库,新任务性能显著提升。代码已开源:https://github.com/aupendu/continual-restore。
原文摘要 · Abstract (English)
Continual learning is an emerging topic in the field of deep learning, where a model is expected to learn continuously for new upcoming tasks without forgetting previous experiences. This field has witnessed numerous advancements, but few works have been attempted in the direction of image restoration. Handling large image sizes and the divergent nature of various degradation poses a unique challenge in the restoration domain. However, existing works require heavily engineered architectural modifications for new task adaptation, resulting in significant computational overhead. Regularization-based methods are unsuitable for restoration, as different restoration challenges require different kinds of feature processing. In this direction, we propose a simple modification of the convolution layer to adapt the knowledge from previous restoration tasks without touching the main backbone architecture. Therefore, it can be seamlessly applied to any deep architecture without any structural modifications. Unlike other approaches, we demonstrate that our model can increase the number of trainable parameters without significantly increasing computational overhead or inference time. Experimental validation demonstrates that new restoration tasks can be introduced without compromising the performance of existing tasks. We also show that performance on new restoration tasks improves by adapting the knowledge from the knowledge base created by previous restoration tasks. The code is available at https://github.com/aupendu/continual-restore.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。