用预训练模型生成点图，提升深度图平滑性，改善3D高斯溅射渲染质量。
Revisiting Depth Representations for Feed-Forward 3D Gaussian Splatting
- 引入PM-Loss，利用预训练Transformer生成点图作为几何正则化信号
- 在多种架构和场景下，显著提升3DGS的渲染质量，尤其改善边界处碎片问题
- 适合关注3D重建与高效视图合成的从业者，无需复杂训练流程
深度图在前馈式3D高斯溅射（3DGS）流程中被广泛用于将图像反投影为3D点云以实现新视角合成。该方法具备训练高效、可利用已知相机位姿及几何估计准确等优势。然而，物体边界处的深度不连续常导致点云断裂或稀疏，降低渲染质量——这是基于深度表示的固有缺陷。为此，我们提出PM-Loss，一种基于预训练Transformer生成点图的新型正则化损失。尽管点图自身精度可能低于深度图，但能有效强制几何平滑性，尤其是在物体边界处。借助优化后的深度图，我们的方法在多种架构与场景中显著提升前馈式3DGS表现，持续获得更优渲染结果。
原文摘要 · Abstract (English)
Depth maps are widely used in feed-forward 3D Gaussian Splatting (3DGS) pipelines by unprojecting them into 3D point clouds for novel view synthesis. This approach offers advantages such as efficient training, the use of known camera poses, and accurate geometry estimation. However, depth discontinuities at object boundaries often lead to fragmented or sparse point clouds, degrading rendering quality -- a well-known limitation of depth-based representations. To tackle this issue, we introduce PM-Loss, a novel regularization loss based on a pointmap predicted by a pre-trained transformer. Although the pointmap itself may be less accurate than the depth map, it effectively enforces geometric smoothness, especially around object boundaries. With the improved depth map, our method significantly improves the feed-forward 3DGS across various architectures and scenes, delivering consistently better rendering results. Our project page: https://aim-uofa.github.io/PMLoss
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。