用高斯混合模型优化图像修复集成,提升精度且无需训练
EnsIR: An Ensemble Algorithm for Image Restoration via Gaussian Mixture Models
- 将图像修复集成建模为高斯混合模型,通过期望最大化算法估计权重
- 在14个基准上优于回归和平均法,在超分、去模糊、去雨任务中均表现更优
- 模型无关且免训练,可无缝增强各类预训练修复模型
图像修复因深度学习发展取得显著进展,但仍面临病态问题导致单模型预测与真实值偏差。集成学习通过融合多个基础模型预测来缓解此问题。现有工作多在模型设计阶段引入集成,而较少关注预训练模型的推理阶段集成。基于回归的方法效率低下,学界与工业界普遍采用平均法作为后训练集成策略。为此,本文将图像修复集成问题重新建模为高斯混合模型(GMM),并采用基于期望最大化(EM)的算法估计集成权重以聚合预测结果。在参考集上估算范围化集成权重,并存储于查找表(LUT)中,实现测试集上的高效集成推理。本方法模型无关且免训练,可无缝集成并提升各类预训练图像修复模型性能。在3类图像修复任务(超分辨率、去模糊、去雨)的14个基准上,该方法持续优于基于回归的方法和平均集成方案。代码与所有估计权重已开源。
原文摘要 · Abstract (English)
Image restoration has experienced significant advancements due to the development of deep learning. Nevertheless, it encounters challenges related to ill-posed problems, resulting in deviations between single model predictions and ground-truths. Ensemble learning, as a powerful machine learning technique, aims to address these deviations by combining the predictions of multiple base models. Most existing works adopt ensemble learning during the design of restoration models, while only limited research focuses on the inference-stage ensemble of pre-trained restoration models. Regression-based methods fail to enable efficient inference, leading researchers in academia and industry to prefer averaging as their choice for post-training ensemble. To address this, we reformulate the ensemble problem of image restoration into Gaussian mixture models (GMMs) and employ an expectation maximization (EM)-based algorithm to estimate ensemble weights for aggregating prediction candidates. We estimate the range-wise ensemble weights on a reference set and store them in a lookup table (LUT) for efficient ensemble inference on the test set. Our algorithm is model-agnostic and training-free, allowing seamless integration and enhancement of various pre-trained image restoration models. It consistently outperforms regression based methods and averaging ensemble approaches on 14 benchmarks across 3 image restoration tasks, including super-resolution, deblurring and deraining. The codes and all estimated weights have been released in Github.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。