用可微分的多视角草图点云重建3D边缘,让结果更准更完整。
SketchSplat: 3D Edge Reconstruction via Differentiable Multi-view Sketch Splatting
- 将3D边缘表示为带控制点、尺度和透明度的参数化草图,通过可微渲染优化
- 在基准CAD数据集上达到顶尖的精度、完整性和紧凑性
- 适合需要高保真3D结构重建的工业设计与逆向工程场景
边缘是描述3D结构信息的基本参数化单元。本文研究从校准的多视角图像中进行参数化3D边缘重建。以往方法通常先从多视角2D边缘图像重建3D点集,再对点集拟合3D边缘,但点集噪声会导致边缘间出现空隙,且拟合结果可能与输入图像不一致,因仅依赖点集。为此,我们提出SketchSplat,一种通过可微分多视角草图点云实现精确、完整、紧凑3D边缘重建的方法。我们将3D边缘表示为草图,即由控制点、尺度和透明度定义的参数化线与曲线。重建过程中,迭代从草图采样高斯点并将其光栅化至2D边缘图像,通过反向传播图像损失优化草图属性。该方法以可微方式连接2D边缘图像与3D边缘,确保3D边缘与2D图像对齐,从而获得更准确、完整的重建结果。此外,我们引入一系列自适应拓扑操作减少冗余边缘,并与草图优化联合使用,提升紧凑性。最后,我们贡献了一个高精度2D边缘检测器,显著提升本方法及现有方法性能。实验表明,该方法在基准CAD数据集上达到当前最优的精度、完整性和紧凑性。
原文摘要 · Abstract (English)
Edges are one of the most basic parametric primitives to describe structural information in 3D. In this paper, we study parametric 3D edge reconstruction from calibrated multi-view images. Previous methods usually reconstruct a 3D edge point set from multi-view 2D edge images, and then fit 3D edges to the point set. However, noise in the point set may cause gaps among fitted edges, and the recovered edges may not align with input multi-view images since the edge fitting depends only on the reconstructed 3D point set. To mitigate these problems, we propose SketchSplat, a method to reconstruct accurate, complete, and compact 3D edges via differentiable multi-view sketch splatting. We represent 3D edges as sketches, which are parametric lines and curves defined by attributes including control points, scales, and opacity. During reconstruction, we iteratively sample Gaussian points from a set of sketches and rasterize the Gaussians onto 2D edge images. Then the gradient of the image loss can be back-propagated to optimize the sketch attributes. Our method bridges 2D edge images and 3D edges in a differentiable manner, which ensures that 3D edges align well with 2D images and leads to accurate and complete results. We also propose a series of adaptive topological operations to reduce redundant edges and apply them along with the sketch optimization, yielding a more compact reconstruction. Finally, we contribute an accurate 2D edge detector that improves the performance of both ours and existing methods. Experiments show that our method achieves state-of-the-art accuracy, completeness, and compactness on a benchmark CAD dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。