用多参考图实现面部部件无缝替换，支持精细人脸定制。
FuseAnyPart: Diffusion-Driven Facial Parts Swapping via Multiple Reference Images
- 在潜在空间中通过掩码融合模块整合多源面部部件
- 通过添加注入模块将特征融合至扩散模型UNet中生成新面孔
- 支持任意面部部件替换，适合角色设计与个性化生成
面部部件替换旨在将源图像中的特定区域转移到目标图像上，同时保持目标图像其余部分不变。现有大多面部替换方法专为整脸替换设计，在处理单个面部部件替换时能力有限或效果不佳，制约了细粒度角色定制。其核心挑战在于如何高效且有效地实现多参考特征的融合。为此，本文提出FuseAnyPart，实现无限制的“任意部件替换”定制。该方法在基于掩码的融合模块中，将来自不同人的面部部件在潜在空间中组合成完整人脸；随后，融合后的特征被送入基于添加的注入模块，嵌入扩散模型的UNet中，生成全新人物形象。大量定性与定量实验验证了FuseAnyPart在性能与鲁棒性上的优势。源代码已开源：https://github.com/Thomas-wyh/FuseAnyPart。
原文摘要 · Abstract (English)
Facial parts swapping aims to selectively transfer regions of interest from the source image onto the target image while maintaining the rest of the target image unchanged. Most studies on face swapping designed specifically for full-face swapping, are either unable or significantly limited when it comes to swapping individual facial parts, which hinders fine-grained and customized character designs. However, designing such an approach specifically for facial parts swapping is challenged by a reasonable multiple reference feature fusion, which needs to be both efficient and effective. To overcome this challenge, FuseAnyPart is proposed to facilitate the seamless "fuse-any-part" customization of the face. In FuseAnyPart, facial parts from different people are assembled into a complete face in latent space within the Mask-based Fusion Module. Subsequently, the consolidated feature is dispatched to the Addition-based Injection Module for fusion within the UNet of the diffusion model to create novel characters. Extensive experiments qualitatively and quantitatively validate the superiority and robustness of FuseAnyPart. Source codes are available at https://github.com/Thomas-wyh/FuseAnyPart.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。