InfoBFR通过信息瓶颈机制，解决真实场景中人脸复原的神经退化问题。
InfoBFR: Real-World Blind Face Restoration via Information Bottleneck
- 基于流形信息瓶颈压缩与高效扩散LoRA补偿优化信息流
- 恢复效果接近85%提升，仅需70ms推理时间、16M参数
- 适合作为通用插件，提升各类人脸复原模型的真实场景表现
盲人脸复原（BFR）因退化模式不确定性而极具挑战。现有方法虽能实现一定恢复效果，但存在固有的神经退化问题，限制了复杂场景下的真实泛化能力。本文提出即插即用框架InfoBFR，有效应对先验偏差、拓扑扭曲、纹理失真及伪影残留等神经退化问题，在多样化的野外和异构场景中实现高泛化性人脸复原。具体而言，基于预训练BFR模型结果，InfoBFR采用流形信息瓶颈（MIB）进行信息压缩，并利用高效扩散LoRA实现信息补偿，完成信息优化。该方法可生成无属性与身份失真的高保真人脸。大量实验表明，InfoBFR在性能上超越当前主流基于GAN与扩散模型的BFR方法，推理耗时约70ms，仅需16M可训练参数，恢复效果提升近85%。未来有望成为通用插件，被多种BFR模型广泛采用以克服神经退化问题。
原文摘要 · Abstract (English)
Blind face restoration (BFR) is a highly challenging problem due to the uncertainty of data degradation patterns. Current BFR methods have realized certain restored productions but with inherent neural degradations that limit real-world generalization in complicated scenarios. In this paper, we propose a plug-and-play framework InfoBFR to tackle neural degradations, e.g., prior bias, topological distortion, textural distortion, and artifact residues, which achieves high-generalization face restoration in diverse wild and heterogeneous scenes. Specifically, based on the results from pre-trained BFR models, InfoBFR considers information compression using manifold information bottleneck (MIB) and information compensation with efficient diffusion LoRA to conduct information optimization. InfoBFR effectively synthesizes high-fidelity faces without attribute and identity distortions. Comprehensive experimental results demonstrate the superiority of InfoBFR over state-of-the-art GAN-based and diffusion-based BFR methods, with around 70ms consumption, 16M trainable parameters, and nearly 85% BFR-boosting. It is promising that InfoBFR will be the first plug-and-play restorer universally employed by diverse BFR models to conquer neural degradations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。