arXiv:2410.18979cs.CVcs.AI2024-10被引 22

动态调整3D高斯数量与分布,提升多视角重建质量

PixelGaussian: Generalizable 3D Gaussian Reconstruction from Arbitrary Views

  • 根据局部几何复杂度动态调整高斯分布和数量
  • 在ACID和RealEstate10K上实现领先性能
  • 适合需要高效多视角重建的研究者

我们提出PixelGaussian,一个高效的前馈框架,用于从任意视角学习可泛化的3D高斯重建。现有方法依赖于统一的像素级高斯表示,为每视图学习固定数量的3D高斯,难以泛化到更多输入视图。而我们的方法基于几何复杂度动态适应高斯分布与数量,实现更高效的表示并显著提升重建质量。具体地,我们引入级联高斯适配器(CGA),通过关键点评分器识别局部几何复杂度,利用上下文感知超网络中的可变形注意力,指导高斯剪枝与分裂,确保复杂区域的精确表示并减少冗余。此外,我们设计了基于Transformer的迭代高斯精修模块,通过直接图像-高斯交互优化表示。PixelGaussian能有效降低输入视图增多时的高斯冗余。我们在大规模ACID和RealEstate10K数据集上进行了广泛实验,结果表明该方法在多种视图数下均达到先进水平,并具有良好的泛化能力。

原文摘要 · Abstract (English)

We propose PixelGaussian, an efficient feed-forward framework for learning generalizable 3D Gaussian reconstruction from arbitrary views. Most existing methods rely on uniform pixel-wise Gaussian representations, which learn a fixed number of 3D Gaussians for each view and cannot generalize well to more input views. Differently, our PixelGaussian dynamically adapts both the Gaussian distribution and quantity based on geometric complexity, leading to more efficient representations and significant improvements in reconstruction quality. Specifically, we introduce a Cascade Gaussian Adapter to adjust Gaussian distribution according to local geometry complexity identified by a keypoint scorer. CGA leverages deformable attention in context-aware hypernetworks to guide Gaussian pruning and splitting, ensuring accurate representation in complex regions while reducing redundancy. Furthermore, we design a transformer-based Iterative Gaussian Refiner module that refines Gaussian representations through direct image-Gaussian interactions. Our PixelGaussian can effectively reduce Gaussian redundancy as input views increase. We conduct extensive experiments on the large-scale ACID and RealEstate10K datasets, where our method achieves state-of-the-art performance with good generalization to various numbers of views. Code: https://github.com/Barrybarry-Smith/PixelGaussian.

3D重建高斯表示多视角动态适配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。