用3D结构引导扩散模型，让美颜更自然且保真。
Diffusion-based Facial Aesthetics Enhancement with 3D Structure Guidance
- 从最近邻参考脸提取美学指导，结合3D人脸建模。
- 生成结果在吸引力提升上优于现有方法，身份保留更好。
- 适合需要高保真美颜的影视、社交应用。
面部美学增强（FAE）旨在调整面部结构与外观以提升吸引力，同时尽可能保持身份一致性。现有方法多采用深度特征或评分引导生成模型进行FAE，虽取得良好效果，但易导致过度美化、身份失真或吸引力提升不足。为在增强美学的同时减少身份损失，本文提出基于最近邻结构引导的扩散模型（NNSG-Diffusion），通过参考最近邻的2D参考脸与输入脸，联合恢复3D人脸模型，提取深度与轮廓信息作为控制信号，引导Stable Diffusion结合ControlNet实现面部美学增强。大量实验表明，该方法在提升面部吸引力的同时显著提升了身份一致性，优于现有相关方法。
原文摘要 · Abstract (English)
Facial Aesthetics Enhancement (FAE) aims to improve facial attractiveness by adjusting the structure and appearance of a facial image while preserving its identity as much as possible. Most existing methods adopted deep feature-based or score-based guidance for generation models to conduct FAE. Although these methods achieved promising results, they potentially produced excessively beautified results with lower identity consistency or insufficiently improved facial attractiveness. To enhance facial aesthetics with less loss of identity, we propose the Nearest Neighbor Structure Guidance based on Diffusion (NNSG-Diffusion), a diffusion-based FAE method that beautifies a 2D facial image with 3D structure guidance. Specifically, we propose to extract FAE guidance from a nearest neighbor reference face. To allow for less change of facial structures in the FAE process, a 3D face model is recovered by referring to both the matched 2D reference face and the 2D input face, so that the depth and contour guidance can be extracted from the 3D face model. Then the depth and contour clues can provide effective guidance to Stable Diffusion with ControlNet for FAE. Extensive experiments demonstrate that our method is superior to previous relevant methods in enhancing facial aesthetics while preserving facial identity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。