无需训练即可实现真实场景下头像无缝替换,支持侧脸和长发等复杂情况。
Zero-Shot Head Swapping in Real-World Scenarios
- 自动生成上下文感知的掩码,适应全头与上半身图像。
- 引入发丝注入模块,精准还原长发细节,提升融合自然度。
- 适用于社交媒体个性化图像生成,尤其适合非正面视角场景。
随着媒体和社交网络对个性化图像需求的增长,将头像图像中的完整头部与身体图像中的躯干进行融合的先进头像替换技术愈发重要。然而,传统方法依赖以面部为中心的裁剪数据且主要针对正面对视图像,限制了其在真实场景中的应用效果。此外,其掩码设计仅优化于特定数据集,在复杂情况下(如长发超出掩码区域)难以实现无缝融合。为克服这些局限并增强对多样化复杂场景的适应能力,本文提出一种新方法HID,可处理从正面到侧向视角的全头与上半身图像,并自动生成上下文感知掩码。我们引入IOMask实现头像与躯干的无缝融合,有效解决集成难题。同时,提出发丝注入模块以更精确捕捉头发细节。实验表明,该方法在头像替换任务中达到当前最佳性能,能在多种挑战性条件下生成视觉一致且逼真的结果。
原文摘要 · Abstract (English)
With growing demand in media and social networks for personalized images, the need for advanced head-swapping techniques, integrating an entire head from the head image with the body from the body image, has increased. However, traditional head swapping methods heavily rely on face-centered cropped data with primarily frontal facing views, which limits their effectiveness in real world applications. Additionally, their masking methods, designed to indicate regions requiring editing, are optimized for these types of dataset but struggle to achieve seamless blending in complex situations, such as when the original data includes features like long hair extending beyond the masked area. To overcome these limitations and enhance adaptability in diverse and complex scenarios, we propose a novel head swapping method, HID, that is robust to images including the full head and the upper body, and handles from frontal to side views, while automatically generating context aware masks. For automatic mask generation, we introduce the IOMask, which enables seamless blending of the head and body, effectively addressing integration challenges. We further introduce the hair injection module to capture hair details with greater precision. Our experiments demonstrate that the proposed approach achieves state-of-the-art performance in head swapping, providing visually consistent and realistic results across a wide range of challenging conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。