用高斯点云统一表示3D物体的形状与材质，支持光照重演。
Luce: Relightable Gaussians for 3D Asset Generation
- 用分模态高斯点云分别建模几何与材质属性
- 在Toys4K上FID提升28%，生成细节更精细
- 适合需要真实材质和可重光照的3D资产生成场景
高保真图像到3D生成需要能同时捕捉几何与外观的3D表示。为支持重光照并集成到标准渲染管线，该表示应包含物理基于渲染（PBR）模态，如反照率、金属度-粗糙度和表面法线。我们提出Luce，一种将几何与PBR材质统一在体素化多模态高斯云中的3D表示，每个模态使用专用高斯基元。变分自编码器将此表示压缩至统一的材质感知潜在空间。修正流变换器从单张图像生成该潜在表示，条件于预训练图像编码器的多层特征，以保留语义上下文和精细空间细节。该潜在表示解码为可重光照的PBR高斯点云及可选的带切线空间法线图的纹理网格。在Toys4K数据集上，Luce实现单图像到3D生成的最先进性能，相较最强基线FID降低28%。我们进一步引入一个由AI生成图像构成的基准测试，Luce在该基准上优于最佳基线，CLIP图像对齐分数达0.8519（基线0.8299）。Luce生成的资产具有可重光照性、几何准确性与材质真实性，能保留文字、标志等精细细节。
原文摘要 · Abstract (English)
High-fidelity image-to-3D generation requires a 3D representation that captures both geometry and appearance. To support relighting and integration into standard rendering pipelines, the representation should include physically based rendering (PBR) modalities such as albedo, metallic-roughness, and surface normals. We propose Luce, a 3D representation that unifies geometry and PBR materials within a voxelized multimodal Gaussian cloud, using dedicated Gaussian primitives for each modality. A variational autoencoder compresses this representation into a unified material-aware latent space. A rectified-flow transformer generates this latent from a single image, conditioned on multi-layer features from a pretrained image encoder that preserve both semantic context and fine spatial detail. The latent then decodes into relightable PBR Gaussians and an optional textured mesh with a tangent-space normal map. On Toys4K, Luce achieves state-of-the-art single-image-to-3D generation, improving FID by 28% over the strongest baseline. We further introduce a benchmark of AI-generated images, on which Luce improves the CLIP image-alignment score over the best baseline (0.8519 vs. 0.8299). Luce generates relightable, geometrically accurate, and materially faithful assets that preserve fine details such as text, logos, and inscriptions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。