arXiv:2410.08181cs.CV2024-10被引 8

仅用一张图生成可随意光照的高保真3D汽车模型

RGM: Reconstructing High-fidelity 3D Car Assets with Relightable 3D-GS Generative Model from a Single Image

  • 用带材质参数的3D高斯点表示物体,支持动态光照
  • 从单张图像重建出带材质和全局光照的3D车模型
  • 适合游戏、自动驾驶等需要真实光照渲染的场景

高质量3D汽车资产生成对视频游戏、自动驾驶和虚拟现实等应用至关重要。现有基于NeRF或3D-GS的3D生成方法通常生成的是固定光照下的朗伯物体,缺乏对材质与全局光照的分离建模,导致生成资产无法在不同光照条件下重光照,限制了下游应用。为此,我们提出一种新型可重光照3D对象生成框架,能自动从单张输入图像快速准确地重建车辆的几何、纹理和材质属性。方法首先构建包含超过1,000个高精度3D车模的大规模合成数据集,采用融合BRDF参数的全局光照可重光照3D高斯原语表示3D物体。在此基础上,设计前馈模型,以图像为输入,输出可重光照3D高斯及全局光照参数。实验表明,本方法生成的3D汽车资产可无缝集成到不同光照条件的道路场景中,对工业应用具有显著实用价值。

原文摘要 · Abstract (English)

The generation of high-quality 3D car assets is essential for various applications, including video games, autonomous driving, and virtual reality. Current 3D generation methods utilizing NeRF or 3D-GS as representations for 3D objects, generate a Lambertian object under fixed lighting and lack separated modelings for material and global illumination. As a result, the generated assets are unsuitable for relighting under varying lighting conditions, limiting their applicability in downstream tasks. To address this challenge, we propose a novel relightable 3D object generative framework that automates the creation of 3D car assets, enabling the swift and accurate reconstruction of a vehicle's geometry, texture, and material properties from a single input image. Our approach begins with introducing a large-scale synthetic car dataset comprising over 1,000 high-precision 3D vehicle models. We represent 3D objects using global illumination and relightable 3D Gaussian primitives integrating with BRDF parameters. Building on this representation, we introduce a feed-forward model that takes images as input and outputs both relightable 3D Gaussians and global illumination parameters. Experimental results demonstrate that our method produces photorealistic 3D car assets that can be seamlessly integrated into road scenes with different illuminations, which offers substantial practical benefits for industrial applications.

3D生成可重光照高斯表示单图像重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。