通过自一致性机制提升无参考图像质量评估模型的鲁棒性
Attention Down-Sampling Transformer, Relative Ranking and Self-Consistency for Blind Image Quality Assessment
- 用Transformer与CNN融合提取局部和全局图像特征
- 在五个数据集上优于现有方法，小数据集表现更优
- 适合关注图像质量评估鲁棒性的研究者
无参考图像质量评估（NR-IQA）旨在不依赖原始图像的情况下估计图像质量。本文提出一种改进机制，通过不同Transformer编码器与CNN联合提取图像的局部与非局部信息。利用Transformer对CNN提取的局部特征进行序列处理，缓解局部性偏差，生成更具代表性的非局部表征。通过基于相对距离排序批次内图像，增强主观与客观评估间的关联性。提出一种自一致性自监督方法，显式应对模型在等变变换下的性能退化问题，确保图像与其水平翻转版本输出一致，从而提升模型鲁棒性。在五个主流图像质量评估数据集上的实证结果表明，所提方法在多数场景下超越现有算法，尤其在小规模数据集上优势显著。代码已公开于https://github.com/mas94/ADTRS。
原文摘要 · Abstract (English)
The no-reference image quality assessment is a challenging domain that addresses estimating image quality without the original reference. We introduce an improved mechanism to extract local and non-local information from images via different transformer encoders and CNNs. The utilization of Transformer encoders aims to mitigate locality bias and generate a non-local representation by sequentially processing CNN features, which inherently capture local visual structures. Establishing a stronger connection between subjective and objective assessments is achieved through sorting within batches of images based on relative distance information. A self-consistency approach to self-supervision is presented, explicitly addressing the degradation of no-reference image quality assessment (NR-IQA) models under equivariant transformations. Our approach ensures model robustness by maintaining consistency between an image and its horizontally flipped equivalent. Through empirical evaluation of five popular image quality assessment datasets, the proposed model outperforms alternative algorithms in the context of no-reference image quality assessment datasets, especially on smaller datasets. Codes are available at \href{https://github.com/mas94/ADTRS}{https://github.com/mas94/ADTRS}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。