用潜在扩散模型生成3D高斯,可高效渲染大场景。
L3DG: Latent 3D Gaussian Diffusion
- 在压缩潜空间中通过扩散模型生成3D高斯,提升效率。
- 可生成房间级场景,支持任意视角实时渲染。
- 适合需要高质量3D生成与大场景建模的开发者。
我们提出L3DG,首个基于潜在3D高斯扩散的生成式3D建模方法。该方法在3D高斯压缩潜空间中进行扩散生成,利用向量量化变分自编码器(VQ-VAE)学习潜空间,采用稀疏卷积架构高效处理房间级场景。这显著降低生成复杂度,实现更高细节的物体级生成,并支持大规模场景扩展。得益于3D高斯表示,生成场景可实现任意视角实时渲染。实验表明,该方法在无条件物体级辐射场生成上视觉质量显著优于现有工作,并成功应用于房间级场景生成。
原文摘要 · Abstract (English)
We propose L3DG, the first approach for generative 3D modeling of 3D Gaussians through a latent 3D Gaussian diffusion formulation. This enables effective generative 3D modeling, scaling to generation of entire room-scale scenes which can be very efficiently rendered. To enable effective synthesis of 3D Gaussians, we propose a latent diffusion formulation, operating in a compressed latent space of 3D Gaussians. This compressed latent space is learned by a vector-quantized variational autoencoder (VQ-VAE), for which we employ a sparse convolutional architecture to efficiently operate on room-scale scenes. This way, the complexity of the costly generation process via diffusion is substantially reduced, allowing higher detail on object-level generation, as well as scalability to large scenes. By leveraging the 3D Gaussian representation, the generated scenes can be rendered from arbitrary viewpoints in real-time. We demonstrate that our approach significantly improves visual quality over prior work on unconditional object-level radiance field synthesis and showcase its applicability to room-scale scene generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。