arXiv:2605.13600cs.CV2026-05

用稀疏代码提升实现高效3D语言高斯点云,训练快400倍、内存省3倍。

Sparse Code Uplifting for Efficient 3D Language Gaussian Splatting

论文配图:Sparse Code Uplifting for Efficient 3D Language Gaussian Splatting
图 1 · 摘自论文原文
  • 在2D图像区域学习稀疏代码,再映射到3D高斯点
  • 通过多视角加权聚合提取关键系数,存储效率提升3倍
  • 适合需要快速语义重建的开放词汇3D场景应用

3D语言高斯点云(3DLGS)通过引入与语言对齐的视觉特征,实现开放词汇的3D场景理解。核心挑战在于如何高效地将高维视觉-语言嵌入与数百万个3D高斯点关联,同时保持文本查询时的高效渲染。现有方法要么直接在高斯点上存储密集特征,导致存储成本高、渲染慢;要么通过重复特征光栅化进行耗时的逐场景优化以学习紧凑表示。目前尚无方法能同时实现快速3D语义重建、高效存储和快速渲染。我们提出SCOUP(稀疏代码提升),通过解耦语言表征学习与3D高斯优化来解决上述问题。不直接在3D空间操作,而是基于2D图像区域的特征学习稀疏码本表示,每个区域关联一组稀疏码本系数。随后通过高斯-像素对应关系,采用加权稀疏聚合将这些系数提升至3D高斯点,每个高斯点累积来自多视角的码本原子系数。最后使用Top-K筛选每高斯点最显著的多视角系数,实现高效存储与快速渲染。相比当前最优方法,我们的方法在训练速度上最高提升400倍,训练期间内存效率提高3倍,且在多个基准测试中达到或超越现有方法的开放词汇查询准确率。

原文摘要 · Abstract (English)

3D Language Gaussian Splatting (3DLGS) augments 3D Gaussian Splatting with language-aligned visual features for open-vocabulary 3D scene understanding. A core challenge is efficiently associating high-dimensional vision-language embeddings with millions of 3D Gaussians while preserving efficient feature rendering for text-based querying. Existing methods either store dense features directly on Gaussians, causing high storage costs and slow rendering, or learn compact representations through expensive per-scene optimization with repeated feature rasterization. No existing method simultaneously achieves fast 3D semantic reconstruction, efficient storage, and fast rendering. We propose SCOUP (Sparse COde UPlifting), which addresses all three by decoupling language representation learning from 3D Gaussian optimization. Rather than working directly in 3D, we learn sparse codebook-based representations entirely using features associated with 2D image regions, associating each region with a sparse set of codebook coefficients. We then uplift these coefficients to 3D Gaussians with our weighted sparse aggregation using Gaussian-to-pixel associations, where each Gaussian accumulates coefficients over codebook atoms across views. Top-$K$ filtering then extracts the most dominant multi-view coefficients per Gaussian, enabling efficient storage and fast rendering. Our method achieves up to $400\times$ training speedup while being $3\times$ more memory efficient during training compared to the state-of-the-art in rendering speed. Across multiple benchmarks, SCOUP matches or outperforms existing methods in open-vocabulary querying accuracy.

3D生成高斯点云语言对齐稀疏编码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。