用三平面表示法直接生成3D高斯点云，实现文本到3D高质量建模
DirectTriGS: Triplane-based Gaussian Splatting Field Representation for 3D Generation
- 采用三平面连续场表示，将离散高斯点云转为图像式结构
- 仅需2D监督即可还原几何与纹理，支持端到端训练
- 可压缩至潜在空间，适配扩散模型生成3D物体
我们提出DirectTriGS，一种基于三平面表示的高斯点云直接生成框架。传统生成方法难以处理高斯点云的复杂结构，而本工作通过三平面表示，将高斯点云编码为类似图像的连续场，同时保留几何与纹理信息。该表示支持通过可微分的TriRenderer重建为点云并渲染成图像，仅需2D监督。由于渲染过程完全可微，损失函数可同时优化纹理与几何。此外，三平面表示可通过变分自编码器（VAE）压缩，进而用于潜在扩散模型生成3D对象。实验表明，该框架在文本到3D任务中能生成高质量几何与渲染结果。
原文摘要 · Abstract (English)
We present DirectTriGS, a novel framework designed for 3D object generation with Gaussian Splatting (GS). GS-based rendering for 3D content has gained considerable attention recently. However, there has been limited exploration in directly generating 3D Gaussians compared to traditional generative modeling approaches. The main challenge lies in the complex data structure of GS represented by discrete point clouds with multiple channels. To overcome this challenge, we propose employing the triplane representation, which allows us to represent Gaussian Splatting as an image-like continuous field. This representation effectively encodes both the geometry and texture information, enabling smooth transformation back to Gaussian point clouds and rendering into images by a TriRenderer, with only 2D supervisions. The proposed TriRenderer is fully differentiable, so that the rendering loss can supervise both texture and geometry encoding. Furthermore, the triplane representation can be compressed using a Variational Autoencoder (VAE), which can subsequently be utilized in latent diffusion to generate 3D objects. The experiments demonstrate that the proposed generation framework can produce high-quality 3D object geometry and rendering results in the text-to-3D task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。