从单图重建穿衣服的人体3D模型，提升骨骼、关节和衣物褶皱精度
MultiGO: Towards Multi-level Geometry Learning for Monocular 3D Textured Human Reconstruction
- 分层级学习：骨骼级增强、关节级补全、褶皱级优化
- 在两个新测试集上优于现有最优方法，关节位置更准，褶皱更清晰
- 适合做高精度人体三维重建的研究者与应用开发者
本文研究从单张图像重建穿衣服的人体3D形态。由于单视角输入固有的模糊性，现有方法依赖预训练的SMPL(-X)估计模型或生成模型提供辅助信息，但仅捕捉人体整体几何结构，忽略具体细节，导致骨骼重建不准、关节位置错误、衣物褶皱不清。为此，我们提出多层级几何学习框架，设计三个关键组件：骨骼级增强模块、关节级增广模块和褶皱级细化模块。具体地，将投影的3D傅里叶特征有效融入高斯重建模型，训练时引入扰动以改善关节深度估计，并通过模拟扩散模型去噪过程来细化人体粗略褶皱。在两个分布外测试集上的大量定量与定性实验表明，本方法性能显著优于当前最优（SOTA）方法。
原文摘要 · Abstract (English)
This paper investigates the research task of reconstructing the 3D clothed human body from a monocular image. Due to the inherent ambiguity of single-view input, existing approaches leverage pre-trained SMPL(-X) estimation models or generative models to provide auxiliary information for human reconstruction. However, these methods capture only the general human body geometry and overlook specific geometric details, leading to inaccurate skeleton reconstruction, incorrect joint positions, and unclear cloth wrinkles. In response to these issues, we propose a multi-level geometry learning framework. Technically, we design three key components: skeleton-level enhancement, joint-level augmentation, and wrinkle-level refinement modules. Specifically, we effectively integrate the projected 3D Fourier features into a Gaussian reconstruction model, introduce perturbations to improve joint depth estimation during training, and refine the human coarse wrinkles by resembling the de-noising process of diffusion model. Extensive quantitative and qualitative experiments on two out-of-distribution test sets show the superior performance of our approach compared to state-of-the-art (SOTA) methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。