跨域人脸风格迁移，精准保留发丝、眼睛等细节。
Domain Generalizable Portrait Style Transfer
- 通过语义对齐与特征融合实现跨域风格迁移
- 在多个测试数据集上优于现有方法，细节更真实
- 适合需要高质量人脸风格化的人像处理场景
本文提出一种具有强泛化能力的人脸风格迁移方法，在不同领域间保持高质量的语义对齐效果，可精准处理头发、眼睛、睫毛、皮肤、嘴唇及背景等区域。我们基于预训练模型和语义适配器建立输入与参考图像间的密集语义对应关系，生成语义对齐的形变参考图。为平衡内容保留与风格化程度，设计了AdaIN-Wavelet变换，将形变参考图的低频信息与输入图像的高频信息在潜在空间融合。同时引入风格适配器提供风格指导。最终利用双条件扩散模型结合ControlNet记录高频细节与风格引导生成结果。大量实验表明该方法性能领先。代码与模型已开源：https://github.com/wangxb29/DGPST。
原文摘要 · Abstract (English)
This paper presents a portrait style transfer method that generalizes well to various different domains while enabling high-quality semantic-aligned stylization on regions including hair, eyes, eyelashes, skins, lips, and background. To this end, we propose to establish dense semantic correspondence between the given input and reference portraits based on a pre-trained model and a semantic adapter, with which we obtain a warped reference semantically aligned with the input. To ensure effective yet controllable style transfer, we devise an AdaIN-Wavelet transform to balance content preservation and stylization by blending low-frequency information of the warped reference with high-frequency information of the input in the latent space. A style adapter is also designed to provide style guidance from the warped reference. With the stylized latent from AdaIN-Wavelet transform, we employ a dual-conditional diffusion model that integrates a ControlNet recording high-frequency information and the style guidance to generate the final result. Extensive experiments demonstrate the superiority of our method. Our code and trained model are available at https://github.com/wangxb29/DGPST.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。