arXiv:2605.16696cs.CV2026-05

用扩散模型修复人脸缺损,还能保持身份一致

Face inpainting with Identity Preserving Latent Diffusion Models

论文配图:Face inpainting with Identity Preserving Latent Diffusion Models
图 1 · 摘自论文原文
  • 基于预训练人脸识别网络提取身份嵌入,控制生成过程
  • 在多个数据集上身份保留效果优于主流扩散模型方法
  • 无需微调,对遮挡、姿态变化更鲁棒,适合真实场景

人脸修复技术旨在视觉上合理恢复缺失或被遮挡的面部区域,但保持身份一致性仍是核心挑战。身份一致对人脸识别、数字取证和人机交互等下游应用至关重要,细微的身份失真可能显著影响性能与可信度。尽管基于扩散模型的生成方法在图像修复方面取得显著进展,但仍难以忠实保留个体特异性面部特征。现有身份感知方法通常依赖昂贵的微调、辅助监督,或对多样遮挡、姿态和面部变化鲁棒性不足。为此,本文提出ID-ControlNet,一种基于潜在扩散模型的身份保持人脸修复框架。该方法基于ControlNet架构,以预训练人脸识别网络提取的面部身份嵌入作为条件,指导扩散过程重建被遮挡区域,同时保证全局面部连贯性和身份保真度。此外,引入身份一致性和三元组损失训练策略,显式约束生成结果与目标身份表示对齐。在CelebA-HQ、FFHQ及新构建的E-Mask数据集上的大量实验表明,ID-ControlNet在身份保留方面显著优于标准扩散模型方法,性能接近当前最优身份感知方法。

原文摘要 · Abstract (English)

Face inpainting techniques recover missing or occluded facial regions in a visually realistic manner, but preserving the identity in the final output remains a fundamental challenge. Identity consistency is crucial for downstream applications such as face recognition, digital forensics, and human-computer interaction, where even subtle identity distortions can significantly degrade performance or trust. Although diffusion-based generative models have recently achieved remarkable progress in image inpainting, they often struggle to faithfully retain individual-specific facial characteristics. On the other hand, existing identity-aware methods typically rely on costly fine-tuning, auxiliary supervision, or exhibit limited robustness to diverse occlusions, poses, and facial variations. To address these limitations, we propose ID-ControlNet, an identity-preserving face inpainting framework built upon latent diffusion models. Based on ControlNet architecture, our approach conditions the diffusion process on facial identity embeddings extracted from a pretrained face recognition network. This design enables reconstruction of occluded facial regions while maintaining global facial coherence and identity fidelity. Furthermore, we introduce an identity consistency and triplet loss training strategy that explicitly enforces alignment between the generated face and the target identity representation. Extensive experiments on CelebA-HQ, FFHQ, and on a new E-Mask dataset demonstrate that ID-ControlNet significantly improves identity preservation over standard diffusion-based inpainting methods, achieving performance comparable to SOTA identity-aware approaches.

人脸修复扩散模型身份保持ControlNet

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。