用人脸修复反推全图退化,实现整体画面高清还原
Face2Scene: Using Facial Degradation as an Oracle for Diffusion-Based Scene Restoration
- 以人脸为感知信标,提取退化特征引导全图恢复
- 单步完成身体与背景修复,显著减少视觉伪影
- 适合图像修复、数字内容重建等场景应用
近期图像修复进展使得基于参考的人脸修复模型(Ref-FR)能够从退化输入中高保真恢复人脸。但这类方法仅关注面部区域,忽略全身及背景的退化,限制了实际应用。而全图修复器常忽视退化线索,导致预测不确定和视觉伪影。本文提出Face2Scene,一种两阶段修复框架:先用Ref-FR重建高质量人脸,再从修复前后的人脸对中提取人脸衍生退化码,编码噪声、模糊、压缩等退化属性,并转化为多尺度退化感知令牌,以此条件扩散模型一次性恢复整幅图像(含身体与背景)。大量实验表明,该方法优于当前最优方法。
原文摘要 · Abstract (English)
Recent advances in image restoration have enabled high-fidelity recovery of faces from degraded inputs using reference-based face restoration models (Ref-FR). However, such methods focus solely on facial regions, neglecting degradation across the full scene, including body and background, which limits practical usability. Meanwhile, full-scene restorers often ignore degradation cues entirely, leading to underdetermined predictions and visual artifacts. In this work, we propose Face2Scene, a two-stage restoration framework that leverages the face as a perceptual oracle to estimate degradation and guide the restoration of the entire image. Given a degraded image and one or more identity references, we first apply a Ref-FR model to reconstruct high-quality facial details. From the restored-degraded face pair, we extract a face-derived degradation code that captures degradation attributes (e.g., noise, blur, compression), which is then transformed into multi-scale degradation-aware tokens. These tokens condition a diffusion model to restore the full scene in a single step, including the body and background. Extensive experiments demonstrate the superior effectiveness of the proposed method compared to state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。