用稀疏视角实现高质量3D渲染,提升点云细节表现
PointGS: Point Attention-Aware Sparse View Synthesis with Gaussian Splatting
- 通过自注意力机制增强点级外观特征表达
- 在少样本下仍保持高画质,优于主流NeRF与3DGS方法
- 适合低资源场景重建,如移动设备实时渲染
3D高斯点阵(3DGS)是一种创新的渲染技术,通过显式三维场景表示,在渲染速度和视觉质量上超越神经辐射场(NeRF)。现有3DGS方法需大量校准视图才能生成一致完整的场景表示。当输入视图稀疏时,3DGS容易过拟合训练视图,导致渲染质量明显下降。为此,我们提出点注意力感知的稀疏视图合成框架,实现从稀疏输入中实时高质量渲染。首先利用最新的立体基础模型估计精确相机位姿并重建稠密点云用于高斯初始化;随后通过采样与聚合多尺度2D外观特征,编码每个3D高斯的颜色属性;为增强点级外观表示,设计基于自注意力机制的点交互网络,使每个高斯点与其最近邻相互作用;最后通过两个轻量级多层感知机(MLPs)将增强特征解码为高斯参数以完成渲染。在多个基准测试上的实验表明,本方法显著优于基于NeRF的方法,在少样本设置下与当前最优3DGS方法相比也具备竞争力。
原文摘要 · Abstract (English)
3D Gaussian splatting (3DGS) is an innovative rendering technique that surpasses the neural radiance field (NeRF) in both rendering speed and visual quality by leveraging an explicit 3D scene representation. Existing 3DGS approaches require a large number of calibrated views to generate a consistent and complete scene representation. When input views are limited, 3DGS tends to overfit the training views, leading to noticeable degradation in rendering quality. To address this limitation, we propose a Point-wise Feature-Aware Gaussian Splatting framework that enables real-time, high-quality rendering from sparse training views. Specifically, we first employ the latest stereo foundation model to estimate accurate camera poses and reconstruct a dense point cloud for Gaussian initialization. We then encode the colour attributes of each 3D Gaussian by sampling and aggregating multiscale 2D appearance features from sparse inputs. To enhance point-wise appearance representation, we design a point interaction network based on a self-attention mechanism, allowing each Gaussian point to interact with its nearest neighbors. These enriched features are subsequently decoded into Gaussian parameters through two lightweight multi-layer perceptrons (MLPs) for final rendering. Extensive experiments on diverse benchmarks demonstrate that our method significantly outperforms NeRF-based approaches and achieves competitive performance under few-shot settings compared to the state-of-the-art 3DGS methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。