通过图神经网络与专家机制,提升图像质量评估的精度与效率。
Life-IQA: Boosting Blind Image Quality Assessment through GCN-enhanced Layer Interaction and MoE-based Feature Decoupling
- 用图卷积增强深层与浅层特征交互,优化信息融合。
- 采用多专家机制解耦不同失真类型的特征,提升评估针对性。
- 在多个基准上达到顶尖性能,兼顾准确率与计算成本。
盲图像质量评估(BIQA)在优化视觉体验中至关重要。现有方法通常融合骨干网络提取的浅层与深层特征,却忽视了它们对质量预测的非均衡贡献。尽管多种视觉编码器被广泛用于BIQA,但有效的质量解码架构仍待深入探索。为此,本文研究浅层与深层特征在BIQA中的贡献,并提出一种基于图卷积增强层间交互与多专家特征解耦的质量解码框架,命名为Life-IQA。具体而言,图卷积增强的层间交互模块以最深层特征为查询,倒数第二层特征为键值,通过交叉注意力实现特征交互;同时,提出基于多专家(MoE)的特征解耦模块,通过专精于特定失真类型或质量维度的专家解耦融合表示。大量实验表明,Life-IQA在准确率与开销之间表现出更优平衡,优于基线Transformer解码器,并在多个BIQA基准上达到当前最优性能。代码已开源。
原文摘要 · Abstract (English)
Blind image quality assessment (BIQA) plays a crucial role in evaluating and optimizing visual experience. Most existing BIQA approaches fuse shallow and deep features extracted from backbone networks, while overlooking the unequal contributions to quality prediction. Moreover, while various vision encoder backbones are widely adopted in BIQA, the effective quality decoding architectures remain underexplored. To address these limitations, this paper investigates the contributions of shallow and deep features to BIQA, and proposes a effective quality feature decoding framework via GCN-enhanced \underline{l}ayer\underline{i}nteraction and MoE-based \underline{f}eature d\underline{e}coupling, termed \textbf{(Life-IQA)}. Specifically, the GCN-enhanced layer interaction module utilizes the GCN-enhanced deepest-layer features as query and the penultimate-layer features as key, value, then performs cross-attention to achieve feature interaction. Moreover, a MoE-based feature decoupling module is proposed to decouple fused representations though different experts specialized for specific distortion types or quality dimensions. Extensive experiments demonstrate that Life-IQA shows more favorable balance between accuracy and cost than a vanilla Transformer decoder and achieves state-of-the-art performance on multiple BIQA benchmarks.The code is available at: \href{https://github.com/TANGLONG2/Life-IQA/tree/main}{\texttt{Life-IQA}}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。