仅用一张图3分钟生成可分解的高质量3D角色,适合游戏影视应用。
StdGEN: Semantic-Decomposed 3D Character Generation from Single Images
- 通过语义感知重建模型,联合恢复几何、颜色与语义信息。
- 生成带身体、衣物、头发等分离组件的3D角色,3分钟完成。
- 支持灵活定制,适用于虚拟现实与动画制作场景。
我们提出StdGEN,一种从单张图像生成语义分解的高质量3D角色的新方法,适用于虚拟现实、游戏和影视等领域。相比以往方法在可分解性、质量与优化时间上的不足,StdGEN具备可分解性、高效性与有效性:可在三分钟内生成包含身体、服装、发型等独立语义组件的精细3D角色。其核心是语义感知大重建模型(S-LRM),基于Transformer的通用化模型,以前馈方式从多视角图像中联合重建几何、颜色与语义。引入可微分的多层语义表面提取方案,从S-LRM生成的混合隐式场中获取网格。同时集成专用高效多视角扩散模型与迭代多层表面精修模块,提升生成质量与可分解性。大量实验表明,在3D动漫角色生成上显著优于现有基线,在几何、纹理与可分解性方面均达领先水平。StdGEN提供即用型语义分解3D角色,支持灵活定制,适用广泛应用场景。项目页:https://stdgen.github.io
原文摘要 · Abstract (English)
We present StdGEN, an innovative pipeline for generating semantically decomposed high-quality 3D characters from single images, enabling broad applications in virtual reality, gaming, and filmmaking, etc. Unlike previous methods which struggle with limited decomposability, unsatisfactory quality, and long optimization times, StdGEN features decomposability, effectiveness and efficiency; i.e., it generates intricately detailed 3D characters with separated semantic components such as the body, clothes, and hair, in three minutes. At the core of StdGEN is our proposed Semantic-aware Large Reconstruction Model (S-LRM), a transformer-based generalizable model that jointly reconstructs geometry, color and semantics from multi-view images in a feed-forward manner. A differentiable multi-layer semantic surface extraction scheme is introduced to acquire meshes from hybrid implicit fields reconstructed by our S-LRM. Additionally, a specialized efficient multi-view diffusion model and an iterative multi-layer surface refinement module are integrated into the pipeline to facilitate high-quality, decomposable 3D character generation. Extensive experiments demonstrate our state-of-the-art performance in 3D anime character generation, surpassing existing baselines by a significant margin in geometry, texture and decomposability. StdGEN offers ready-to-use semantic-decomposed 3D characters and enables flexible customization for a wide range of applications. Project page: https://stdgen.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。