arXiv:2607.28300cs.CVcs.AI2026-07

用单目视频实现轻量级3D场景语义解析,大幅降低内存占用。

MonoVoc: Decoupling Geometry and Semantics for Lightweight Monocular Open-Vocabulary 3D Gaussians

论文配图:MonoVoc: Decoupling Geometry and Semantics for Lightweight Monocular Open-Vocabulary 3D Gaussians
图 1 · 摘自论文原文
  • 分离几何与语义建模,先独立重建3D结构再轻量注入语义
  • 相比顶尖方法内存减少一个数量级,保持高保真渲染和分割精度
  • 适合日常单目视频直接做开放词汇3D检索与问答

开放词汇3D场景理解对下一代交互系统至关重要,使用户能通过自然语言直观查询和导航重建环境。然而,现有3D高斯框架常受限于多视角采集要求、高昂的场景特异性优化成本,以及存储密集型语言特征带来的巨大内存开销。本文提出一种无需训练的新流程,从根本上重构该范式:显式解耦3D几何重建与语义融合。仅需标准单目视频输入,即可高效生成紧凑、可解释且全可搜索的对象级语义高斯地图。不将复杂语言嵌入嵌入映射过程,而是独立提取几何,并通过轻量模块化后处理框架实现语义定位。在Replica数据集上的大量实验表明,该解耦架构保持了强渲染保真度和竞争性分割准确率。关键在于,用模块化对象级语义嵌入替代每高斯点的密集存储,相较当前最优基线实现一个数量级的内存压缩。为从日常单目视频直接进行开放词汇3D检索与问答提供了高效、可扩展且实用的解决方案。

原文摘要 · Abstract (English)

Open vocabulary 3D scene understanding is essential for next-generation interactive systems, empowering users to intuitively query and navigate reconstructed environments using natural language. However, current 3D Gaussian frameworks are often bottlenecked by restrictive multiview capture requirements, costly scene-specific optimization, and the massive memory overhead of storing dense language features. We present a novel, training-free pipeline that fundamentally reimagines this paradigm by explicitly decoupling 3D geometric reconstruction from semantic integration. Given a standard monocular video sequence as input, our method efficiently outputs a compact, highly interpretable, and fully searchable object-level semantic Gaussian map. Rather than entangling heavy language embeddings within the mapping loop, we extract geometry independently and ground semantics through a lightweight, modular post-processing framework. Extensive evaluations on the Replica dataset demonstrate that this decoupled architecture preserves strong rendering fidelity and competitive segmentation accuracy. Crucially, by replacing dense per-Gaussian storage with modular, object-level semantic embeddings, our approach delivers an order-of-magnitude reduction in memory usage compared to SOTA baselines. This provides a highly efficient, scalable, and practical solution for open-vocabulary 3D retrieval and question answering directly from everyday monocular video.

3D高斯单目重建语义解析轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。