arXiv:2503.09294cs.CV2025-03被引 4

用图像质量先验提升人脸修复清晰度,突破数据限制。

IQPFR: An Image Quality Prior for Blind Face Restoration and Beyond

  • 引入无参考图像质量评估作为质量先验,指导修复方向。
  • 通过双分支码本学习,分离通用结构与高清特征。
  • 质量条件化框架可无缝接入现有修复模型,提升效果。

盲人脸修复(BFR)旨在将低质量(LQ)人脸图像恢复为高质量(HQ)输出。传统方法依赖真实标签(GT)数据学习特征表示,但GT数据固有的质量局限使修复性能受限于训练数据的平均质量水平,难以达到最优视觉质量。为此,我们提出一种新框架,利用来自无参考图像质量评估(NR-IQA)模型的图像质量先验(IQP),引导修复过程向最大可能的质量标准逼近。该方法通过两项关键创新实现:(1)在码本学习阶段,设计双分支码本架构,将特征提取分为通用结构成分与高清特定属性,全面捕捉常见与高质量人脸特征;(2)在码本查找阶段,采用基于Transformer的质量条件化框架,以NR-IQA生成的质量分数作为动态条件信号,驱动修复向最高可行质量演进。该评分条件范式支持对现有BFR模型进行即插即用式增强,无需修改原始结构。此外,我们提出基于离散表示的质量优化策略,避免连续隐空间中常见的过优化伪影。大量实验表明,本方法在多个基准上优于现有最先进技术,且在集成到已有模型后仍保持稳定提升。代码将开源。

原文摘要 · Abstract (English)

Blind Face Restoration (BFR) addresses the challenge of reconstructing degraded low-quality (LQ) facial images into high-quality (HQ) outputs. Conventional approaches predominantly rely on learning feature representations from ground-truth (GT) data; however, inherent imperfections in GT datasets constrain restoration performance to the mean quality level of the training data, rather than attaining maximally attainable visual quality. To overcome this limitation, we propose a novel framework that incorporates an Image Quality Prior (IQP) derived from No-Reference Image Quality Assessment (NR-IQA) models to guide the restoration process toward optimal HQ reconstructions. Our methodology synergizes this IQP with a learned codebook prior through two critical innovations: (1) During codebook learning, we devise a dual-branch codebook architecture that disentangles feature extraction into universal structural components and HQ-specific attributes, ensuring comprehensive representation of both common and high-quality facial characteristics. (2) In the codebook lookup stage, we implement a quality-conditioned Transformer-based framework. NR-IQA-derived quality scores act as dynamic conditioning signals to steer restoration toward the highest feasible quality standard. This score-conditioned paradigm enables plug-and-play enhancement of existing BFR architectures without modifying the original structure. We also formulate a discrete representation-based quality optimization strategy that circumvents over-optimization artifacts prevalent in continuous latent space approaches. Extensive experiments demonstrate that our method outperforms state-of-the-art techniques across multiple benchmarks. Besides, our quality-conditioned framework demonstrates consistent performance improvements when integrated with prior BFR models. The code will be released.

人脸修复图像质量先验建模NR-IQA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。