arXiv:2412.16552cs.CVcs.AI2024-12AAAI被引 12

通过掩码策略与迭代优化,提升真实人脸超分的一致性与质量。

Diffusion Prior Interpolation for Flexibility Real-World Face Super-Resolution

  • 基于人脸结构设计强弱约束掩码,分阶段控制生成过程。
  • 引入条件修正器实现条件与样本的相互优化,提升精度。
  • 无需微调即可适配预训练模型,适合实际应用部署。

扩散模型在生成建模中处于领先地位。由于训练成本高,许多工作通过微调或基于先验的方法利用预训练扩散模型的强大表征进行下游任务,如人脸超分辨率(FSR)。然而,仅依赖先验而无监督训练难以满足判别任务的像素级精度要求。尽管基于先验的方法可实现高保真和高质量结果,但保持一致性仍是重大挑战。本文提出一种掩码策略与迭代优化方法,称为扩散先验插值(DPI),通过基于人脸结构特征在不同采样阶段施加条件与约束来增强一致性。此外,我们提出条件修正器(CRT),建立双向后验采样流程,通过条件与样本的相互精炼提升FSR性能。DPI能有效平衡一致性与多样性,并可无缝集成至预训练模型。在合成与真实数据集上的大量实验,以及人脸识别的一致性验证表明,DPI优于当前最优方法。代码已开源。

原文摘要 · Abstract (English)

Diffusion models represent the state-of-the-art in generative modeling. Due to their high training costs, many works leverage pre-trained diffusion models' powerful representations for downstream tasks, such as face super-resolution (FSR), through fine-tuning or prior-based methods. However, relying solely on priors without supervised training makes it challenging to meet the pixel-level accuracy requirements of discrimination task. Although prior-based methods can achieve high fidelity and high-quality results, ensuring consistency remains a significant challenge. In this paper, we propose a masking strategy with strong and weak constraints and iterative refinement for real-world FSR, termed Diffusion Prior Interpolation (DPI). We introduce conditions and constraints on consistency by masking different sampling stages based on the structural characteristics of the face. Furthermore, we propose a condition Corrector (CRT) to establish a reciprocal posterior sampling process, enhancing FSR performance by mutual refinement of conditions and samples. DPI can balance consistency and diversity and can be seamlessly integrated into pre-trained models. In extensive experiments conducted on synthetic and real datasets, along with consistency validation in face recognition, DPI demonstrates superiority over SOTA FSR methods. The code is available at \url{https://github.com/JerryYann/DPI}.

人脸超分扩散模型一致性优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。