arXiv:2411.17163cs.CV2024-11CVPR被引 41

一拍即合:1步生成高清人脸,又真又像

OSDFace: One-Step Diffusion Model for Face Restoration

  • 用视觉嵌入器提取低质人脸特征,生成精准提示
  • 融合人脸识别损失,身份一致率显著提升
  • 结合GAN引导,图像更自然真实,适合实时修复

扩散模型在人脸修复中表现优异,但多步推理计算开销大,限制了实际应用。现有方法常难以生成和谐、逼真且身份一致的人脸图像。本文提出OSDFace,一种新型的一步扩散模型。首先设计视觉表示嵌入器(VRE),通过视觉分词器处理低质人脸,并利用向量量化字典生成视觉提示。同时引入基于人脸识别的面部身份损失,强化身份一致性。此外,采用生成对抗网络(GAN)作为引导模型,促进重建结果与真实图像分布对齐。实验表明,OSDFace在视觉质量与定量指标上均超越当前最优方法,生成高保真、自然且身份一致的人脸图像。代码与模型将开源。

原文摘要 · Abstract (English)

Diffusion models have demonstrated impressive performance in face restoration. Yet, their multi-step inference process remains computationally intensive, limiting their applicability in real-world scenarios. Moreover, existing methods often struggle to generate face images that are harmonious, realistic, and consistent with the subject's identity. In this work, we propose OSDFace, a novel one-step diffusion model for face restoration. Specifically, we propose a visual representation embedder (VRE) to better capture prior information and understand the input face. In VRE, low-quality faces are processed by a visual tokenizer and subsequently embedded with a vector-quantized dictionary to generate visual prompts. Additionally, we incorporate a facial identity loss derived from face recognition to further ensure identity consistency. We further employ a generative adversarial network (GAN) as a guidance model to encourage distribution alignment between the restored face and the ground truth. Experimental results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics, generating high-fidelity, natural face images with high identity consistency. The code and model will be released at https://github.com/jkwang28/OSDFace.

人脸修复扩散模型一步生成身份一致

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。