发现深度网络中的稀疏子网可兼具高效与高精度,提升图像重建性能。
Chasing Better Deep Image Priors between Over- and Under-parameterization
- 从过参数化网络中挖掘稀疏子网作为新型图像先验
- 在紧凑模型下超越深解码器,保持原模型效果
- 适用于多种图像修复任务,且具备强迁移能力
深度神经网络(DNN)通常作为过参数化的图像先验(DIP),用于各类图像逆问题的正则化。与此同时,研究者也提出了极简的欠参数化先验(如deep decoder),虽精度略低但效率极高。这促使我们思考:是否存在介于两者之间的更优解?受彩票理论(LTH)启发,本文提出“彩票图像先验”(LIP):一个过参数化DNN中存在稀疏子网,可独立训练,在图像逆问题中表现与原网络相当。实验验证了该假设:在显著稀疏度下仍能定位有效子网。这些子网在模型规模相近时显著优于deep decoder,且几乎完全保留原始DIP性能,并具备跨图像和任务的强迁移性。此外,我们将LIP扩展至压缩感知图像重建,使用预训练GAN生成器作为先验,同样验证其有效性。据我们所知,这是首次将LTH成功应用于图像逆问题或图像先验场景。
原文摘要 · Abstract (English)
Deep Neural Networks (DNNs) are well-known to act as over-parameterized deep image priors (DIP) that regularize various image inverse problems. Meanwhile, researchers also proposed extremely compact, under-parameterized image priors (e.g., deep decoder) that are strikingly competent for image restoration too, despite a loss of accuracy. These two extremes push us to think whether there exists a better solution in the middle: between over- and under-parameterized image priors, can one identify "intermediate" parameterized image priors that achieve better trade-offs between performance, efficiency, and even preserving strong transferability? Drawing inspirations from the lottery ticket hypothesis (LTH), we conjecture and study a novel "lottery image prior" (LIP) by exploiting DNN inherent sparsity, stated as: given an over-parameterized DNN-based image prior, it will contain a sparse subnetwork that can be trained in isolation, to match the original DNN's performance when being applied as a prior to various image inverse problems. Our results validate the superiority of LIPs: we can successfully locate the LIP subnetworks from over-parameterized DIPs at substantial sparsity ranges. Those LIP subnetworks significantly outperform deep decoders under comparably compact model sizes (by often fully preserving the effectiveness of their over-parameterized counterparts), and they also possess high transferability across different images as well as restoration task types. Besides, we also extend LIP to compressive sensing image reconstruction, where a pre-trained GAN generator is used as the prior (in contrast to untrained DIP or deep decoder), and confirm its validity in this setting too. To our best knowledge, this is the first time that LTH is demonstrated to be relevant in the context of inverse problems or image priors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。