用点云中间表示，实现快速高保真单图3D重建
SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Images
- 两阶段设计：先用轻量点扩散模型生成稀疏点云，再结合图像生成精细网格
- 推理速度仅0.7秒，重建精度超越现有最先进方法
- 支持用户交互编辑，适合需要高效高质3D重建的场景
我们研究单图像3D物体重建问题。近期工作分为回归建模和生成建模两条路径：回归方法能高效推断可见表面，但难以处理遮挡区域；生成方法通过建模分布更好处理不确定区域，但计算开销大且生成结果常与可见表面错位。本文提出SPAR3D，一种新型两阶段方法，兼顾两者优势。第一阶段使用轻量级点扩散模型生成稀疏3D点云，采样速度快；第二阶段结合采样点云与输入图像生成高细节网格。该两阶段设计在保持概率建模能力的同时，实现高计算效率和高输出保真度。以点云为中间表示，还支持用户交互编辑。在多个数据集上的评估表明，SPAR3D性能优于先前最先进方法，推理速度仅为0.7秒。
原文摘要 · Abstract (English)
We study the problem of single-image 3D object reconstruction. Recent works have diverged into two directions: regression-based modeling and generative modeling. Regression methods efficiently infer visible surfaces, but struggle with occluded regions. Generative methods handle uncertain regions better by modeling distributions, but are computationally expensive and the generation is often misaligned with visible surfaces. In this paper, we present SPAR3D, a novel two-stage approach aiming to take the best of both directions. The first stage of SPAR3D generates sparse 3D point clouds using a lightweight point diffusion model, which has a fast sampling speed. The second stage uses both the sampled point cloud and the input image to create highly detailed meshes. Our two-stage design enables probabilistic modeling of the ill-posed single-image 3D task while maintaining high computational efficiency and great output fidelity. Using point clouds as an intermediate representation further allows for interactive user edits. Evaluated on diverse datasets, SPAR3D demonstrates superior performance over previous state-of-the-art methods, at an inference speed of 0.7 seconds. Project page with code and model: https://spar3d.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。