用图像提示优化3D试穿，实现更逼真的虚拟试衣效果。
IPVTON: Image-based 3D Virtual Try-on with Image Prompt Adapter
- 通过图像提示适配器将服装特征融入扩散模型先验。
- 利用掩码引导聚焦试穿区域，避免非目标区域干扰。
- 结合伪轮廓约束保持身份形状，提升几何与纹理精度。
给定一张人物和一件服装的独立图像，基于图像的3D虚拟试穿方法旨在重建一个真实呈现该人物穿着目标服装的3D人体模型。本文提出IPVTON，一种新型基于图像的3D虚拟试穿框架。IPVTON采用带图像提示的得分蒸馏采样，优化混合式3D人体表示，通过图像提示适配器将目标服装特征融入扩散先验。为避免对非目标区域的干扰，采用掩码引导的图像提示嵌入，聚焦于试穿区域。此外，借助ControlNet生成的伪轮廓施加几何约束，确保着装3D人体模型在保留源身份形状的同时准确穿戴目标服装。大量定性与定量实验表明，IPVTON在基于图像的3D虚拟试穿任务中优于现有方法，尤其在几何与纹理表现上更优。
原文摘要 · Abstract (English)
Given a pair of images depicting a person and a garment separately, image-based 3D virtual try-on methods aim to reconstruct a 3D human model that realistically portrays the person wearing the desired garment. In this paper, we present IPVTON, a novel image-based 3D virtual try-on framework. IPVTON employs score distillation sampling with image prompts to optimize a hybrid 3D human representation, integrating target garment features into diffusion priors through an image prompt adapter. To avoid interference with non-target areas, we leverage mask-guided image prompt embeddings to focus the image features on the try-on regions. Moreover, we impose geometric constraints on the 3D model with a pseudo silhouette generated by ControlNet, ensuring that the clothed 3D human model retains the shape of the source identity while accurately wearing the target garments. Extensive qualitative and quantitative experiments demonstrate that IPVTON outperforms previous methods in image-based 3D virtual try-on tasks, excelling in both geometry and texture.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。