从单图生成物理一致的高质量3D模型,支持真实光照渲染。
GraphicsDreamer: Image to 3D Generation with Physical Consistency
- 融合PBR光照方程的跨域扩散模型,同步预测多视角图像与材质。
- 生成模型具备真实光照响应能力,纹理细节可靠且可直接导入引擎。
- 支持拓扑优化与快速UV展开,适合游戏/影视工业级应用。
近年来,高效自动的3D AI生成内容(AIGC)方法正加速将人类想象力转化为复杂3D结构。然而,自动化3D内容生成在工业应用中仍显著滞后,原因在于3D建模需满足高精度几何、优良拓扑及基于物理的渲染(PBR)等高标准。为缩小生成结果与艺术家期望的差距,我们提出GraphicsDreamer,一种从单张图像生成高度可用3D网格的方法。为更准确捕捉几何与材质细节,我们将PBR光照方程融入跨域扩散模型,同时预测多视角颜色、法线、深度图及PBR材质。在几何融合阶段,持续施加PBR约束,确保生成3D对象具备可靠的纹理细节,支持真实光照重渲染。此外,方法集成拓扑优化与快速UV展开功能,使生成的3D资产可无缝导入图形引擎。大量实验表明,相比先前方法,该模型能在合理时间成本内生成高质量3D资产。
原文摘要 · Abstract (English)
Recently, the surge of efficient and automated 3D AI-generated content (AIGC) methods has increasingly illuminated the path of transforming human imagination into complex 3D structures. However, the automated generation of 3D content is still significantly lags in industrial application. This gap exists because 3D modeling demands high-quality assets with sharp geometry, exquisite topology, and physically based rendering (PBR), among other criteria. To narrow the disparity between generated results and artists' expectations, we introduce GraphicsDreamer, a method for creating highly usable 3D meshes from single images. To better capture the geometry and material details, we integrate the PBR lighting equation into our cross-domain diffusion model, concurrently predicting multi-view color, normal, depth images, and PBR materials. In the geometry fusion stage, we continue to enforce the PBR constraints, ensuring that the generated 3D objects possess reliable texture details, supporting realistic relighting. Furthermore, our method incorporates topology optimization and fast UV unwrapping capabilities, allowing the 3D products to be seamlessly imported into graphics engines. Extensive experiments demonstrate that our model can produce high quality 3D assets in a reasonable time cost compared to previous methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。