单图生成3D模型，速度快质量高，支持多视角一致渲染。
Direct and Explicit 3D Generation from a Single Image
- 用扩散模型+3D高斯特征直接生成带纹理的显式几何体
- 多视角一致性通过对极注意力实现，生成速度远超现有方法
- 适合需要快速高质量3D生成的应用场景
当前图像转3D方法计算成本高，难以扩展至高分辨率输出。为此，我们提出一种新框架，通过复用Stable Diffusion模型，结合多视角2D深度与RGB图像及3D高斯特征，直接生成显式表面几何与纹理。在U-Net中引入深度分支以实现高效、高质量的多视角跨域生成，并在潜在空间到像素的解码器中加入对极注意力，确保像素级多视角一致性。通过将生成的深度像素反投影至3D空间，构建结构化3D表示，既可采用高斯溅射渲染，也可提取为高质量网格。此外，引入新型视图合成损失进一步提升性能。大量实验表明，本方法在几何与纹理质量上均优于现有基线，且生成速度显著更快。
原文摘要 · Abstract (English)
Current image-to-3D approaches suffer from high computational costs and lack scalability for high-resolution outputs. In contrast, we introduce a novel framework to directly generate explicit surface geometry and texture using multi-view 2D depth and RGB images along with 3D Gaussian features using a repurposed Stable Diffusion model. We introduce a depth branch into U-Net for efficient and high quality multi-view, cross-domain generation and incorporate epipolar attention into the latent-to-pixel decoder for pixel-level multi-view consistency. By back-projecting the generated depth pixels into 3D space, we create a structured 3D representation that can be either rendered via Gaussian splatting or extracted to high-quality meshes, thereby leveraging additional novel view synthesis loss to further improve our performance. Extensive experiments demonstrate that our method surpasses existing baselines in geometry and texture quality while achieving significantly faster generation time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。