arXiv:2602.24043cs.CV2026-02TPAMI

用扩散模型重建衣服3D形状,支持单图和视频,效果逼真。

Spatio-Temporal Garment Reconstruction Using Diffusion Mapping via Pattern Coordinates

  • 在2D贴图空间用扩散模型学衣服形状先验,再映射到3D
  • 单图和视频都能重建,松身衣也能保持细节和动态真实
  • 适合虚拟试衣、角色动画等下游应用

从单目图像和视频中重建3D着装人体是虚拟试衣、角色创建和混合现实的基础问题。尽管人体恢复已有进展,但松身衣物的几何重建仍是开放挑战。本文提出统一框架,实现高保真3D衣物重建,适用于单图与视频序列。方法结合隐式缝合图案(ISP)与生成扩散模型,在2D UV空间学习表达性强的衣物形状先验。利用这些先验,构建映射模型,建立图像像素、UV坐标与3D几何间的对应关系,实现单图下的精确、细致的衣物重建。进一步通过时空扩散机制与测试时引导,扩展至动态重建,确保长时序一致性。引入基于解析投影的约束,保留可见区域的图像对齐几何,同时在遮挡区域实现时间上连贯的补全。虽仅在合成布料数据上训练,该方法在真实图像上仍表现出色,优于现有方法,尤其在紧身与松身衣物上均能保持精细几何与自然动态,支持纹理编辑、服装重定向与动画等下游应用。

原文摘要 · Abstract (English)

Reconstructing 3D clothed humans from monocular images and videos is a fundamental problem with applications in virtual try-on, avatar creation, and mixed reality. Despite significant progress in human body recovery, accurately reconstructing garment geometry, particularly for loose-fitting clothing, remains an open challenge. We propose a unified framework for high-fidelity 3D garment reconstruction from both single images and video sequences. Our approach combines Implicit Sewing Patterns (ISP) with a generative diffusion model to learn expressive garment shape priors in 2D UV space. Leveraging these priors, we introduce a mapping model that establishes correspondences between image pixels, UV pattern coordinates, and 3D geometry, enabling accurate and detailed garment reconstruction from single images. We further extend this formulation to dynamic reconstruction by introducing a spatio-temporal diffusion scheme with test-time guidance to enforce long-range temporal consistency. We also develop analytic projection-based constraints that preserve image-aligned geometry in visible regions while enforcing coherent completion in occluded areas over time. Although trained exclusively on synthetically simulated cloth data, our method generalizes well to real-world imagery and consistently outperforms existing approaches on both tight- and loose-fitting garments. The reconstructed garments preserve fine geometric detail while exhibiting realistic dynamic motion, supporting downstream applications such as texture editing, garment retargeting, and animation.

3D重建扩散模型虚拟试衣衣物建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。