从单图分离重建人体与衣物的3D形态,解决遮挡难题。
DeClotH: Decomposable 3D Cloth and Human Body Reconstruction from a Single Image
- 用人体和衣物3D模板做几何先验,缓解相互遮挡问题。
- 引入专用布料扩散模型,提升衣物外观重建精度。
- 适合关注3D服装建模、虚拟试衣的研究者和开发者。
现有从单张图像重建穿衣服人体的方法通常将衣着人体视为单一整体,未区分衣物与人体。为此,我们提出DeClotH,可分别重建3D衣物与人体。该任务因衣物与人体间严重遮挡而长期未被充分探索,导致几何与纹理推断困难。尽管近期3D人体重建利用文本到图像扩散模型取得显著进展,但直接应用于本问题常导致错误引导,尤其在衣物重建方面。为此,我们设计两项核心机制:首先,借助衣物与人体的3D模板作为正则化,提供强几何先验,防止遮挡带来的错误重建;其次,引入专门设计的布料扩散模型,提供衣物外观的上下文信息,增强3D衣物重建效果。定性和定量实验表明,所提方法在人体与衣物重建上均表现优异。更多结果见 https://hygenie1228.github.io/DeClotH/。
原文摘要 · Abstract (English)
Most existing methods of 3D clothed human reconstruction from a single image treat the clothed human as a single object without distinguishing between cloth and human body. In this regard, we present DeClotH, which separately reconstructs 3D cloth and human body from a single image. This task remains largely unexplored due to the extreme occlusion between cloth and the human body, making it challenging to infer accurate geometries and textures. Moreover, while recent 3D human reconstruction methods have achieved impressive results using text-to-image diffusion models, directly applying such an approach to this problem often leads to incorrect guidance, particularly in reconstructing 3D cloth. To address these challenges, we propose two core designs in our framework. First, to alleviate the occlusion issue, we leverage 3D template models of cloth and human body as regularizations, which provide strong geometric priors to prevent erroneous reconstruction by the occlusion. Second, we introduce a cloth diffusion model specifically designed to provide contextual information about cloth appearance, thereby enhancing the reconstruction of 3D cloth. Qualitative and quantitative experiments demonstrate that our proposed approach is highly effective in reconstructing both 3D cloth and the human body. More qualitative results are provided at https://hygenie1228.github.io/DeClotH/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。