用自上而下的方法，5%的高斯点实现高效3D特征场重建。
CF3: Compact and Fast 3D Feature Fields
- 先融合多视角2D特征，再训练每高斯点的自编码器。
- 仅用5%高斯点即达与基线相当的3D特征场质量。
- 自适应稀疏化保留几何细节，适合实时3D建模应用。
3D Gaussian Splatting (3DGS) 开始融入来自2D基础模型的丰富信息。然而，大多数方法依赖自下而上的优化过程，将原始2D特征视为真实值，导致计算成本增加。本文提出一种自上而下的3D高斯特征场构建框架，称为CF3。首先对多视图2D特征进行快速加权融合，并结合预训练的高斯点。该方法使我们能够直接在提升后的特征上训练每高斯点的自编码器，而非在2D域训练。结果，自编码器更好地匹配特征分布。更重要的是，引入一种自适应稀疏化方法，在剪枝和合并冗余高斯点的同时优化特征属性，构建出高效且保留几何细节的表示。本方法在仅使用5%高斯点的情况下，仍能获得与Feature-3DGS相当的3D特征场质量。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has begun incorporating rich information from 2D foundation models. However, most approaches rely on a bottom-up optimization process that treats raw 2D features as ground truth, incurring increased computational costs. We propose a top-down pipeline for constructing compact and fast 3D Gaussian feature fields, namely, CF3. We first perform a fast weighted fusion of multi-view 2D features with pre-trained Gaussians. This approach enables training a per-Gaussian autoencoder directly on the lifted features, instead of training autoencoders in the 2D domain. As a result, the autoencoder better aligns with the feature distribution. More importantly, we introduce an adaptive sparsification method that optimizes the Gaussian attributes of the feature field while pruning and merging the redundant Gaussians, constructing an efficient representation with preserved geometric details. Our approach achieves a competitive 3D feature field using as little as 5% of the Gaussians compared to Feature-3DGS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。