迭代优化代码解码,提升真实场景去雾效果。
Iterative Predictor-Critic Code Decoding for Real-World Image Dehazing
- 利用前轮解码结果指导下一循环的代码预测,逐步优化。
- 在多个真实去雾数据集上达到最优性能,显著优于现有方法。
- 适合需要高精度去雾的应用,如自动驾驶、遥感图像处理。
我们提出一种名为 IPC-Dehaze 的新型迭代预测-批判代码解码框架,用于真实世界图像去雾。该方法借助预训练 VQGAN 中高质量的代码本先验,突破以往基于代码本方法仅支持一次性解码的限制。通过利用前一迭代获得的高质量代码来指导下一迭代中代码预测器的输出,提升了代码预测准确率并保障了去雾性能的稳定性。核心思想源于两个观察:1)雾霾图像的退化程度随雾霾密度和场景深度变化;2)清晰区域为恢复密集雾霾区域提供关键线索。然而,在后续迭代中对代码进行渐进式优化存在挑战,主要在于难以判断每轮应保留或替换哪些代码。为此,我们引入代码批判模块(Code-Critic),用于捕捉代码间的相互关系,并根据掩码得分重新采样代码,得分越高表示越可能被剔除,从而保留更准确的代码并预测困难区域。大量实验表明,本方法在真实世界去雾任务中显著优于当前最先进的方法。
原文摘要 · Abstract (English)
We propose a novel Iterative Predictor-Critic Code Decoding framework for real-world image dehazing, abbreviated as IPC-Dehaze, which leverages the high-quality codebook prior encapsulated in a pre-trained VQGAN. Apart from previous codebook-based methods that rely on one-shot decoding, our method utilizes high-quality codes obtained in the previous iteration to guide the prediction of the Code-Predictor in the subsequent iteration, improving code prediction accuracy and ensuring stable dehazing performance. Our idea stems from the observations that 1) the degradation of hazy images varies with haze density and scene depth, and 2) clear regions play crucial cues in restoring dense haze regions. However, it is non-trivial to progressively refine the obtained codes in subsequent iterations, owing to the difficulty in determining which codes should be retained or replaced at each iteration. Another key insight of our study is to propose Code-Critic to capture interrelations among codes. The Code-Critic is used to evaluate code correlations and then resample a set of codes with the highest mask scores, i.e., a higher score indicates that the code is more likely to be rejected, which helps retain more accurate codes and predict difficult ones. Extensive experiments demonstrate the superiority of our method over state-of-the-art methods in real-world dehazing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。