用贝叶斯方法动态建模场景语义，实现低内存高保真3D重建。
Bayesian Fields: Task-driven Open-Set Semantic Gaussian Splatting
- 基于任务需求动态定义物体粒度，避免人工阈值
- 利用视觉-语言模型特性进行贝叶斯更新，融合多视角信息
- 3D高斯聚类实现高效对象提取与内存优化
开放集语义映射需解决两个问题：一是确定场景表征的合适粒度（如物体如何定义），二是将多个2D观测中的语义知识融合为整体3D重建——理想情况是高保真且低内存。现有方法通常通过手动调参的相似性阈值合并语义相近的原始体素，而本文认为物体粒度应依赖任务目标。为此，提出任务驱动的语义映射方法。针对多视角融合问题，不同于传统平均视觉嵌入向量的做法，本文利用视觉-语言基础模型的内在性质，采用贝叶斯更新策略聚合多视图观测。结果是名为Bayesian Fields的新方法，结合3D高斯表示与任务相关聚类，既能实现高保真物体表达，又支持密集场景重建，同时便于3D对象提取并降低内存占用。代码已开源：https://github.com/MIT-SPARK/Bayesian-Fields。
原文摘要 · Abstract (English)
Open-set semantic mapping requires (i) determining the correct granularity to represent the scene (e.g., how should objects be defined), and (ii) fusing semantic knowledge across multiple 2D observations into an overall 3D reconstruction -ideally with a high-fidelity yet low-memory footprint. While most related works bypass the first issue by grouping together primitives with similar semantics (according to some manually tuned threshold), we recognize that the object granularity is task-dependent, and develop a task-driven semantic mapping approach. To address the second issue, current practice is to average visual embedding vectors over multiple views. Instead, we show the benefits of using a probabilistic approach based on the properties of the underlying visual-language foundation model, and leveraging Bayesian updating to aggregate multiple observations of the scene. The result is Bayesian Fields, a task-driven and probabilistic approach for open-set semantic mapping. To enable high-fidelity objects and a dense scene representation, Bayesian Fields uses 3D Gaussians which we cluster into task-relevant objects, allowing for both easy 3D object extraction and reduced memory usage. We release Bayesian Fields open-source at https: //github.com/MIT-SPARK/Bayesian-Fields.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。