用图像质量先验提升人脸修复清晰度,突破数据限制。
IQPFR: An Image Quality Prior for Blind Face Restoration and Beyond
- 引入无参考图像质量评估作为质量先验,指导修复方向。
- 通过双分支码本学习,分离通用结构与高清特征。
- 质量条件化框架可无缝接入现有修复模型,提升效果。
盲人脸修复(BFR)旨在将低质量(LQ)人脸图像恢复为高质量(HQ)输出。传统方法依赖真实标签(GT)数据学习特征表示,但GT数据固有的质量局限使修复性能受限于训练数据的平均质量水平,难以达到最优视觉质量。为此,我们提出一种新框架,利用来自无参考图像质量评估(NR-IQA)模型的图像质量先验(IQP),引导修复过程向最大可能的质量标准逼近。该方法通过两项关键创新实现:(1)在码本学习阶段,设计双分支码本架构,将特征提取分为通用结构成分与高清特定属性,全面捕捉常见与高质量人脸特征;(2)在码本查找阶段,采用基于Transformer的质量条件化框架,以NR-IQA生成的质量分数作为动态条件信号,驱动修复向最高可行质量演进。该评分条件范式支持对现有BFR模型进行即插即用式增强,无需修改原始结构。此外,我们提出基于离散表示的质量优化策略,避免连续隐空间中常见的过优化伪影。大量实验表明,本方法在多个基准上优于现有最先进技术,且在集成到已有模型后仍保持稳定提升。代码将开源。
原文摘要 · Abstract (English)
Blind Face Restoration (BFR) addresses the challenge of reconstructing degraded low-quality (LQ) facial images into high-quality (HQ) outputs. Conventional approaches predominantly rely on learning feature representations from ground-truth (GT) data; however, inherent imperfections in GT datasets constrain restoration performance to the mean quality level of the training data, rather than attaining maximally attainable visual quality. To overcome this limitation, we propose a novel framework that incorporates an Image Quality Prior (IQP) derived from No-Reference Image Quality Assessment (NR-IQA) models to guide the restoration process toward optimal HQ reconstructions. Our methodology synergizes this IQP with a learned codebook prior through two critical innovations: (1) During codebook learning, we devise a dual-branch codebook architecture that disentangles feature extraction into universal structural components and HQ-specific attributes, ensuring comprehensive representation of both common and high-quality facial characteristics. (2) In the codebook lookup stage, we implement a quality-conditioned Transformer-based framework. NR-IQA-derived quality scores act as dynamic conditioning signals to steer restoration toward the highest feasible quality standard. This score-conditioned paradigm enables plug-and-play enhancement of existing BFR architectures without modifying the original structure. We also formulate a discrete representation-based quality optimization strategy that circumvents over-optimization artifacts prevalent in continuous latent space approaches. Extensive experiments demonstrate that our method outperforms state-of-the-art techniques across multiple benchmarks. Besides, our quality-conditioned framework demonstrates consistent performance improvements when integrated with prior BFR models. The code will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。