用特征大小提升图像质量评估,不需训练就能超越现有方法
Beyond Cosine Similarity: Magnitude-Aware CLIP for No-Reference Image Quality Assessment
- 引入特征幅度作为新质量线索,与语义相似度互补
- 通过Box-Cox变换标准化特征分布,降低语义敏感性
- 自适应融合机制根据置信度动态加权,适合无监督场景
近期工作将对比语言-图像预训练(CLIP)模型用于无参考图像质量评估(NR-IQA),通过计算图像嵌入与文本提示(如'好照片'或'差照片')之间的余弦相似度实现。然而,这种语义相似度忽略了关键但未被充分探索的线索:CLIP图像特征的幅度,我们实证发现其与感知质量有强相关性。本文提出一种新型自适应融合框架,将余弦相似度与幅度感知的质量线索结合。首先提取图像特征的绝对值,并应用Box-Cox变换对特征分布进行统计归一化,以缓解语义敏感性。所得标量摘要作为语义归一化的辅助线索,补充基于提示的匹配。为有效整合双线索,设计了置信度引导的融合方案,按各自相对强度自适应加权。在多个基准IQA数据集上的实验表明,该方法在无需任何任务特定训练的情况下,持续优于标准CLIP基线和当前最优方法。
原文摘要 · Abstract (English)
Recent efforts have repurposed the Contrastive Language-Image Pre-training (CLIP) model for No-Reference Image Quality Assessment (NR-IQA) by measuring the cosine similarity between the image embedding and textual prompts such as "a good photo" or "a bad photo." However, this semantic similarity overlooks a critical yet underexplored cue: the magnitude of the CLIP image features, which we empirically find to exhibit a strong correlation with perceptual quality. In this work, we introduce a novel adaptive fusion framework that complements cosine similarity with a magnitude-aware quality cue. Specifically, we first extract the absolute CLIP image features and apply a Box-Cox transformation to statistically normalize the feature distribution and mitigate semantic sensitivity. The resulting scalar summary serves as a semantically-normalized auxiliary cue that complements cosine-based prompt matching. To integrate both cues effectively, we further design a confidence-guided fusion scheme that adaptively weighs each term according to its relative strength. Extensive experiments on multiple benchmark IQA datasets demonstrate that our method consistently outperforms standard CLIP-based IQA and state-of-the-art baselines, without any task-specific training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。