用扩散模型实现饰品高保真虚拟试戴，解决细节丢失问题
Shining Yourself: High-Fidelity Ornaments Virtual Try-on with Diffusion Model
- 通过迭代估计佩戴掩码提升饰品与人体对齐精度
- 在大规模姿态和尺度变化下仍保持身份一致性和外观真实感
- 适合需要精细饰品试穿的电商与元宇宙应用
尽管基于扩散模型的服装鞋类虚拟试穿已受关注，但项链、手链、戒指、耳环等饰品的虚拟试穿仍鲜有研究。由于多数饰品具有复杂微小图案和重复几何结构，在饰品与模特之间存在显著姿态和尺度差异时，保持身份与外观一致性极为困难。本文提出饰品虚拟试穿任务，并提出一种方法以增强几何与外观保真度。具体而言，我们设计了一种迭代方案，在去噪过程中同步估计精准的佩戴掩码，提升饰品与人体的对齐效果；为保留结构细节，进一步通过隐式正则化注意力层，将参考饰品掩码映射至佩戴掩码。实验表明，该方法能成功将参考图像中的饰品穿戴至目标模特，有效处理大尺度与姿态差异，同时保持身份一致性并生成逼真视觉效果。
原文摘要 · Abstract (English)
While virtual try-on for clothes and shoes with diffusion models has gained attraction, virtual try-on for ornaments, such as bracelets, rings, earrings, and necklaces, remains largely unexplored. Due to the intricate tiny patterns and repeated geometric sub-structures in most ornaments, it is much more difficult to guarantee identity and appearance consistency under large pose and scale variances between ornaments and models. This paper proposes the task of virtual try-on for ornaments and presents a method to improve the geometric and appearance preservation of ornament virtual try-ons. Specifically, we estimate an accurate wearing mask to improve the alignments between ornaments and models in an iterative scheme alongside the denoising process. To preserve structure details, we further regularize attention layers to map the reference ornament mask to the wearing mask in an implicit way. Experimental results demonstrate that our method successfully wears ornaments from reference images onto target models, handling substantial differences in scale and pose while preserving identity and achieving realistic visual effects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。