针对超高清图像质量评估难题,提出多视角融合的高效算法。
Assessing UHD Image Quality from Aesthetics, Distortions, and Saliency
- 分三路提取全局美感、局部失真和显著区域特征
- 在UHD-IQA数据集上性能最优且计算量最低
- 适合超高清图像质检与竞赛应用
超高清图像(通常分辨率等于或高于4K)给高效图像质量评估(IQA)带来挑战,因全分辨率输入导致计算复杂度高,而常用预处理方法如缩放或裁剪又会造成细节丢失。为此,我们设计了一个多分支深度神经网络,从三个角度评估超高清图像质量:全局美学特征、局部技术失真和显著内容感知。具体地,从下采样的低分辨率图像中提取美学特征,保留全局美感但损失高频纹理;通过网格小块采样策略从超高清图像中裁剪出碎片图像以测量技术失真;检测并裁剪显著区域,提取其质量感知特征。采用Swin Transformer Tiny作为主干网络提取三路特征,再通过两层MLP进行特征拼接并回归为质量分数。使用均方误差(MSE)优化预测精度,使用保真度损失优化预测单调性。实验结果表明,该模型在UHD-IQA数据集上表现最佳,同时保持最低计算复杂度,并在ECCV AIM 2024超高清图像质量评估挑战赛中获得第一名。代码已开源。
原文摘要 · Abstract (English)
UHD images, typically with resolutions equal to or higher than 4K, pose a significant challenge for efficient image quality assessment (IQA) algorithms, as adopting full-resolution images as inputs leads to overwhelming computational complexity and commonly used pre-processing methods like resizing or cropping may cause substantial loss of detail. To address this problem, we design a multi-branch deep neural network (DNN) to assess the quality of UHD images from three perspectives: global aesthetic characteristics, local technical distortions, and salient content perception. Specifically, aesthetic features are extracted from low-resolution images downsampled from the UHD ones, which lose high-frequency texture information but still preserve the global aesthetics characteristics. Technical distortions are measured using a fragment image composed of mini-patches cropped from UHD images based on the grid mini-patch sampling strategy. The salient content of UHD images is detected and cropped to extract quality-aware features from the salient regions. We adopt the Swin Transformer Tiny as the backbone networks to extract features from these three perspectives. The extracted features are concatenated and regressed into quality scores by a two-layer multi-layer perceptron (MLP) network. We employ the mean square error (MSE) loss to optimize prediction accuracy and the fidelity loss to optimize prediction monotonicity. Experimental results show that the proposed model achieves the best performance on the UHD-IQA dataset while maintaining the lowest computational complexity, demonstrating its effectiveness and efficiency. Moreover, the proposed model won first prize in ECCV AIM 2024 UHD-IQA Challenge. The code is available at https://github.com/sunwei925/UIQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。