用秒级生成3D场景的扩散模型，让文本直接变3D高保真结果。
Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generation
- 在潜在空间中以多视角像素对齐方式生成3D高斯点云。
- 基于预训练图文模型微调，仅需少量调整即可实现高效生成。
- 引入RGB-D潜空间分离外观与几何，提升生成质量与速度。
本文提出Prometheus，一种用于秒级文本到3D场景生成的3D感知潜在扩散模型，适用于物体和场景级别。我们将3D场景生成建模为在潜在扩散框架内进行多视角、前馈式、像素对齐的3D高斯生成。为保证泛化能力，模型基于预训练的文生图模型构建，仅做最小调整，并利用大量单视图与多视图数据集进行训练。此外，我们引入RGB-D潜空间以解耦外观与几何信息，实现高效前馈生成3D高斯，且保持更高保真度与几何精度。大量实验表明，该方法在前馈3D高斯重建及文本到3D生成任务中均表现优异。
原文摘要 · Abstract (English)
In this work, we introduce Prometheus, a 3D-aware latent diffusion model for text-to-3D generation at both object and scene levels in seconds. We formulate 3D scene generation as multi-view, feed-forward, pixel-aligned 3D Gaussian generation within the latent diffusion paradigm. To ensure generalizability, we build our model upon pre-trained text-to-image generation model with only minimal adjustments, and further train it using a large number of images from both single-view and multi-view datasets. Furthermore, we introduce an RGB-D latent space into 3D Gaussian generation to disentangle appearance and geometry information, enabling efficient feed-forward generation of 3D Gaussians with better fidelity and geometry. Extensive experimental results demonstrate the effectiveness of our method in both feed-forward 3D Gaussian reconstruction and text-to-3D generation. Project page: https://freemty.github.io/project-prometheus/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。