用自然语言在超大规模3D场景中快速精准找物体,无需训练
QueryGaussian: Scalable and Training-Free Open-Vocabulary 3D Instance Retrieval

- 用预训练视觉模型解析提示词,逐个匹配3D实例
- 推理速度提升180倍,显存占用降低70%以上
- 可在消费级显卡上运行城市级场景,适合实时应用
通过自然语言提示从大规模3D场景中高效检索特定实例,仍是多媒体分析中的重大挑战。现有方法多采用“场景级嵌入”范式,需将高维语义特征压缩到每个3D基元,导致内存与计算开销随场景复杂度线性增长,在城市级环境中极易引发显存溢出(OOM)。为此,我们提出QueryGaussian,一种无需训练的可扩展开放词汇3D实例检索框架。不同于整体语义压缩,QueryGaussian采用实例级查询机制,将语义理解与几何表示解耦。具体地,利用预训练2D视觉模型解析用户提示,并通过并行最大权重关联策略将分割掩码投影至3D,保证语义-视觉一致性。为缓解投影歧义,引入带有多阶段自适应密度聚类的时间融合模块。实验表明,QueryGaussian不仅达到顶尖方法的精度,还实现决定性效率提升:GPU显存使用减少超70%,推理速度加快180倍。关键在于,该框架可在消费级硬件上处理包含数千万高斯点的城市级场景,实现快速实例检索。
原文摘要 · Abstract (English)
Efficiently retrieving specific 3D instances from large-scale scenes via natural language prompts remains a formidable challenge in multimedia analysis. Existing approaches predominantly follow a "scene-level embedding" paradigm, which requires distilling high-dimensional semantic features into every 3D primitive. This strategy suffers from a fundamental architectural bottleneck: memory and computational costs scale linearly with scene complexity, inevitably triggering out-of-memory (OOM) failures in city-scale environments. To address this barrier, we propose QueryGaussian, a training-free framework for expeditious and scalable open-vocabulary 3D instance retrieval. Unlike holistic semantic distillation, QueryGaussian employs an instance-level query mechanism that decouples semantic understanding from geometric representation. Specifically, we leverage pre-trained 2D vision models to interpret user prompts and lift segmentation masks into 3D via a concurrent maximum-weight association strategy, ensuring semantic-visual consistency. To mitigate projection ambiguity, we introduce a temporal fusion module with multi-stage adaptive density clustering. Experimental results demonstrate that QueryGaussian not only matches the accuracy of state-of-the-art methods but also delivers a decisive efficiency leap, reducing GPU memory usage by over 70% and accelerating inference by 180x. Crucially, QueryGaussian enables expeditious instance retrieval on city-scale scenes containing tens of millions of Gaussians using consumer-grade hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。