单图重建穿衣人体,通过几何与纹理协同提升真实感。
MultiGO++: Monocular 3D Clothed Human Reconstruction via Geometry-Texture Collaboration
- 多源纹理合成生成超1.5万张3D人体贴图数据。
- 区域感知几何提取+傅里叶编码,有效缓解模态差异。
- 双路重建网络融合几何纹理,生成高保真3D人体模型。
单目3D穿衣人体重建旨在从单张图像生成完整且逼真的带纹理3D角色。现有方法通常依赖多视角监督与标注的几何先验进行训练,推理时则由预训练网络从单目输入估计这些先验。此类方法受三大限制:纹理上因缺乏训练数据,几何上因外部先验不准确,系统上因单一模态监督存在偏差,导致重建效果不佳。为此,本文提出新框架MultiGO++,实现有效的几何-纹理协同。其核心包括:(1) 多源纹理合成策略,构建超过15,000个3D带纹理人体扫描,提升复杂场景下纹理质量估计性能;(2) 区域感知形状提取模块,分区域提取并交互特征以获取几何信息,结合傅里叶几何编码器,减轻模态差距,实现高效几何学习;(3) 双重建U-Net,利用几何-纹理协同特征精炼并生成高保真带纹理3D人体网格。在两个基准数据集及大量真实场景测试中,本方法均优于当前最优技术。
原文摘要 · Abstract (English)
Monocular 3D clothed human reconstruction aims to generate a complete and realistic textured 3D avatar from a single image. Existing methods are commonly trained under multi-view supervision with annotated geometric priors, and during inference, these priors are estimated by the pre-trained network from the monocular input. These methods are constrained by three key limitations: texturally by unavailability of training data, geometrically by inaccurate external priors, and systematically by biased single-modality supervision, all leading to suboptimal reconstruction. To address these issues, we propose a novel reconstruction framework, named MultiGO++, which achieves effective systematic geometry-texture collaboration. It consists of three core parts: (1) A multi-source texture synthesis strategy that constructs 15,000+ 3D textured human scans to improve the performance on texture quality estimation in challenge scenarios; (2) A region-aware shape extraction module that extracts and interacts features of each body region to obtain geometry information and a Fourier geometry encoder that mitigates the modality gap to achieve effective geometry learning; (3) A dual reconstruction U-Net that leverages geometry-texture collaborative features to refine and generate high-fidelity textured 3D human meshes. Extensive experiments on two benchmarks and many in-the-wild cases show the superiority of our method over state-of-the-art approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。