解决3D高斯点云中语言特征的可见性错配与多视角噪声问题。
Visibility-Aware Language Aggregation for Open-Vocabulary Segmentation in 3D Gaussian Splatting
- 按光线可见性加权，筛选有效高斯点。
- 多视角语言特征通过余弦空间几何中位数融合。
- 提升跨视角一致性，适合开放词汇3D分割任务。
近期将2D图像中的开放词汇语言特征蒸馏到3D高斯模型受到关注。现有方法虽实现良好语言交互，但存在两大问题：渲染像素对应的背景高斯点与主导前景点获得相同特征，且多视角语言嵌入受视图特异性噪声影响导致不一致。本文提出可见性感知语言聚合（VALA），一种轻量高效的方法：计算每条光线的边际贡献，并通过可见性门控保留仅可见高斯点；同时在余弦空间采用流式加权几何中位数融合多视角噪声特征。该方法以快速、低内存方式生成鲁棒且视角一致的语言特征嵌入。在多个基准数据集上，VALA持续优于现有方法，显著提升开放词汇定位与分割性能。更多结果见 https://vala3d.github.io
原文摘要 · Abstract (English)
Recently, distilling open-vocabulary language features from 2D images into 3D Gaussians has attracted significant attention. Although existing methods achieve impressive language-based interactions of 3D scenes, we observe two fundamental issues: background Gaussians contributing negligibly to a rendered pixel get the same feature as the dominant foreground ones, and multi-view inconsistencies due to view-specific noise in language embeddings. We introduce Visibility-Aware Language Aggregation (VALA), a lightweight yet effective method that computes marginal contributions for each ray and applies a visibility-aware gate to retain only visible Gaussians. Moreover, we propose a streaming weighted geometric median in cosine space to merge noisy multi-view features. Our method yields a robust, view-consistent language feature embedding in a fast and memory-efficient manner. VALA improves open-vocabulary localization and segmentation across reference datasets, consistently surpassing existing works. More results are available at https://vala3d.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。