用参考人脸提升修复图像的身份一致性，效果更优。
Reference-Guided Identity Preserving Face Restoration
- 融合高低层信息构建参考人脸的综合上下文表示
- 提出新损失函数，解决身份信息学习效率低的问题
- 无需训练即可支持多参考输入，适合实际应用
在基于扩散模型的图像修复中，保持人脸身份一致是一项关键且持续的挑战。尽管参考人脸提供了可行路径，但现有方法未能充分发挥其潜力。本文提出一种新方法，最大化参考人脸的利用价值，实现更好的人脸修复与身份保留。主要贡献包括：1）复合上下文（Composite Context），融合参考人脸的多层次（高低层）信息，提供比单一表示更丰富的指导；2）硬样本身份损失（Hard Example Identity Loss），利用参考人脸改进现有身份损失中的学习效率问题；3）一种推理时无需训练即可适应多参考输入的方法。所提方法在FFHQ-Ref和CelebA-Ref-Test等基准上显著提升了修复质量，并实现了当前最优的身份保留效果，持续优于先前工作。
原文摘要 · Abstract (English)
Preserving face identity is a critical yet persistent challenge in diffusion-based image restoration. While reference faces offer a path forward, existing reference-based methods often fail to fully exploit their potential. This paper introduces a novel approach that maximizes reference face utility for improved face restoration and identity preservation. Our method makes three key contributions: 1) Composite Context, a comprehensive representation that fuses multi-level (high- and low-level) information from the reference face, offering richer guidance than prior singular representations. 2) Hard Example Identity Loss, a novel loss function that leverages the reference face to address the identity learning inefficiencies found in the existing identity loss. 3) A training-free method to adapt the model to multi-reference inputs during inference. The proposed method demonstrably restores high-quality faces and achieves state-of-the-art identity preserving restoration on benchmarks such as FFHQ-Ref and CelebA-Ref-Test, consistently outperforming previous work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。