仅用单目视频和点击，实现3D场景的开放词汇分割。
GeoSAM-3D: Geodesic Prompt Propagation for Open-Vocabulary 3D Scene Segmentation from Monocular Video
- 通过地缘距离传播提示，而非欧氏距离
- 在曲面处保持分割连续性，减少跨物体泄漏
- 适合交互式3D标注，轻量级部署
开放词汇3D场景分割通常依赖RGB-D视频、校准多视角图像或重建网格。GeoSAM-3D研究更轻量的设定：用户上传一段单目视频，在某一帧中点击或命名物体，即可获得高斯场景上的3D掩码传播结果。该方法结合冻结的图像与视频基础模型、单目3D高斯点云重建，以及基于高斯中心点的可微图-地缘传播核。核心设计是采用热核距离在重构场景图上进行提示传播，而非在3D空间中使用欧氏最近邻。这能更好地保持曲面附近的连续性，并降低相邻但不相连物体间的泄漏。本文介绍了代码库状态、geosam3d.propagate中实现的数学核、从Segment Anything掩码训练的特征头，以及代码库中已有的验证内容。评估协议分离了实现验证、图传播质量、泄漏控制与交互延迟四个维度。
原文摘要 · Abstract (English)
Open-vocabulary 3D scene segmentation usually assumes RGB-D video, calibrated multi-view imagery, or a reconstructed mesh. GeoSAM-3D studies a lighter setting: a user uploads a short monocular video, clicks or names an object in one frame, and receives a propagated 3D mask over a Gaussian scene. The implementation combines frozen image and video foundation models with a monocular 3D Gaussian Splatting reconstruction and a differentiable graph-geodesic propagation kernel over Gaussian centroids. The central design choice is to propagate prompts by heat-kernel distance on the reconstructed scene graph, rather than by Euclidean nearest neighbors in 3D. This preserves continuity around curved surfaces and reduces leakage across nearby but disconnected objects. This paper describes the repository state, the mathematical kernel implemented in geosam3d.propagate, the feature head trained from Segment Anything masks, and the validation already present in the codebase. The evaluation protocol separates implementation validation, graph propagation quality, leakage control, and interactive latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。