arXiv:2504.18215cs.CV2025-04被引 2

端到端重建单图中穿衣服的人体3D模型,无需中间几何表示。

Unify3D: An Augmented Holistic End-to-end Monocular 3D Human Reconstruction via Anatomy Shaping and Twins Negotiating

  • 用两个模态不同的U-Net交互增强重建,直接从2D图生成3D avatar。
  • 在两个测试集上优于现有最先进方法,复杂场景表现更稳定。
  • 自建1.5万+三维人体扫描数据,提升对漫画等复杂输入的适应性。

单目3D穿衣人体重建旨在从一张图像生成完整的3D虚拟角色。现有方法通常依赖前置模型提供显式的几何表示,再基于该表示与输入图像联合建模,但受限于前置模型且忽视重建任务的整体性。本文提出新范式,将人体重建视为整体过程,采用端到端网络直接从2D图像预测3Davatar,省去显式中间几何表示。在此基础上,提出双核心组件:解剖结构塑造提取模块(Anatomy Shaping Extraction),捕捉符合人体解剖特性的隐式形状特征;双塔协商重建U-Net(Twins Negotiating Reconstruction U-Net),通过两组不同模态的U-Net间特征交互提升重建质量。此外,提出漫画数据增强策略,并构建超过1.5万条3D人体扫描数据以提升模型在复杂输入下的性能。在两个测试集及大量真实场景下进行的实验表明,本方法显著优于现有最先进方法。演示视频见:https://e2e3dgsrecon.github.io/e2e3dgsrecon/。

原文摘要 · Abstract (English)

Monocular 3D clothed human reconstruction aims to create a complete 3D avatar from a single image. To tackle the human geometry lacking in one RGB image, current methods typically resort to a preceding model for an explicit geometric representation. For the reconstruction itself, focus is on modeling both it and the input image. This routine is constrained by the preceding model, and overlooks the integrity of the reconstruction task. To address this, this paper introduces a novel paradigm that treats human reconstruction as a holistic process, utilizing an end-to-end network for direct prediction from 2D image to 3D avatar, eliminating any explicit intermediate geometry display. Based on this, we further propose a novel reconstruction framework consisting of two core components: the Anatomy Shaping Extraction module, which captures implicit shape features taking into account the specialty of human anatomy, and the Twins Negotiating Reconstruction U-Net, which enhances reconstruction through feature interaction between two U-Nets of different modalities. Moreover, we propose a Comic Data Augmentation strategy and construct 15k+ 3D human scans to bolster model performance in more complex case input. Extensive experiments on two test sets and many in-the-wild cases show the superiority of our method over SOTA methods. Our demos can be found in : https://e2e3dgsrecon.github.io/e2e3dgsrecon/.

3D重建端到端人体建模图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。