arXiv:2412.06753cs.CV2024-12International Conf…被引 15

单步生成+共享注意力,快速还原人脸身份细节

InstantRestore: Single-Step Personalized Face Restoration with Shared-Image Attention

  • 用单步扩散模型与共享注意力机制,实现高效修复
  • 仅需4张参考图,一次前向传播即完成修复
  • 适合作为实时人脸修复系统,尤其注重身份保留

人脸图像修复旨在提升退化面部图像质量,同时应对退化类型多样、实时处理需求以及最关键的身份特征保持难题。现有方法常因处理速度慢、修复效果不佳,尤其在严重退化下难以准确重建细粒度身份特征。为此,我们提出InstantRestore,一种新型框架,利用单步图像扩散模型与注意力共享机制,实现快速且个性化的脸像修复。此外,该方法引入新颖的地标注意力损失,通过对齐关键面部特征点来优化注意力图,增强身份保持能力。推理时,给定一张退化输入和约4张参考图像,InstantRestore仅需一次网络前向传播即可实现接近实时性能。相比依赖完整扩散过程或逐身份模型调优的先前方法,InstantRestore具备可扩展性,适用于大规模应用。大量实验表明,InstantRestore在质量和速度上均优于现有方法,是身份保持型人脸修复的理想选择。

原文摘要 · Abstract (English)

Face image restoration aims to enhance degraded facial images while addressing challenges such as diverse degradation types, real-time processing demands, and, most crucially, the preservation of identity-specific features. Existing methods often struggle with slow processing times and suboptimal restoration, especially under severe degradation, failing to accurately reconstruct finer-level identity details. To address these issues, we introduce InstantRestore, a novel framework that leverages a single-step image diffusion model and an attention-sharing mechanism for fast and personalized face restoration. Additionally, InstantRestore incorporates a novel landmark attention loss, aligning key facial landmarks to refine the attention maps, enhancing identity preservation. At inference time, given a degraded input and a small (~4) set of reference images, InstantRestore performs a single forward pass through the network to achieve near real-time performance. Unlike prior approaches that rely on full diffusion processes or per-identity model tuning, InstantRestore offers a scalable solution suitable for large-scale applications. Extensive experiments demonstrate that InstantRestore outperforms existing methods in quality and speed, making it an appealing choice for identity-preserving face restoration.

人脸修复扩散模型单步生成身份保持

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。