用CLIP模型从全局与局部视角盲评去雾图像质量
CLIP-DQA: Blindly Evaluating Dehazed Images from Global and Local Perspectives Using CLIP
- 结合图像全局与局部特征，利用CLIP模型进行质量评估
- 在两个真实数据集上优于现有方法，预测更准确
- 适合需要无参考图像质量评估的研究者使用
盲去雾图像质量评估（BDQA）旨在无参考情况下准确预测去雾图像的视觉质量，对算法评估、比较和优化至关重要。现有基于学习的BDQA方法虽取得显著进展，但因高质量数据集规模小而受限。本文提出将大规模图文预训练的CLIP模型迁移至BDQA任务。受人类视觉系统基于层次特征理解图像的启发，我们以去雾图像的全局与局部信息作为CLIP输入，并通过提示学习微调其视觉分支与语言分支，以精确映射输入特征到质量分数。在两个真实去雾图像质量评估数据集上的实验表明，所提方法CLIP-DQA在质量预测精度上超越现有方法。代码已开源：https://github.com/JunFu1995/CLIP-DQA。
原文摘要 · Abstract (English)
Blind dehazed image quality assessment (BDQA), which aims to accurately predict the visual quality of dehazed images without any reference information, is essential for the evaluation, comparison, and optimization of image dehazing algorithms. Existing learning-based BDQA methods have achieved remarkable success, while the small scale of DQA datasets limits their performance. To address this issue, in this paper, we propose to adapt Contrastive Language-Image Pre-Training (CLIP), pre-trained on large-scale image-text pairs, to the BDQA task. Specifically, inspired by the fact that the human visual system understands images based on hierarchical features, we take global and local information of the dehazed image as the input of CLIP. To accurately map the input hierarchical information of dehazed images into the quality score, we tune both the vision branch and language branch of CLIP with prompt learning. Experimental results on two authentic DQA datasets demonstrate that our proposed approach, named CLIP-DQA, achieves more accurate quality predictions over existing BDQA methods. The code is available at https://github.com/JunFu1995/CLIP-DQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。