提出DISC框架,实现大场景开放集语义地图的实时高精度构建。
DISC: Dense Integrated Semantic Context for Large-Scale Open-Set Semantic Mapping
- 单次扫描提取语义特征,避免传统裁剪带来的延迟与误差。
- 在Replica和ScanNet上达到最新零样本方法最优的语义准确率与查询召回率。
- 适合需实时语义理解的机器人导航与大型建筑环境建图任务。
开放集语义地图支持语言驱动的机器人感知,但现有基于实例的方法受限于上下文缺失和计算成本高昂的裁剪特征提取。为此,我们提出DISC(密集集成语义上下文),采用新颖的单次、距离加权特征提取机制。通过直接从视觉变换器中间层获取高保真CLIP嵌入,该方法消除了传统图像裁剪带来的延迟和域偏移问题,生成纯净且掩码对齐的语义表示。为在大规模连续建图中充分利用这些特征,DISC基于全GPU加速架构,将周期性离线处理替换为精确的实时体素级实例优化。我们在标准基准(Replica、ScanNet)及新构建的大规模建图数据集HM3DSEM(基于Habitat-Matterport 3D)上评估了其在多层复杂建筑中的可扩展性。大量实验表明,DISC显著超越当前最先进的零样本方法,在语义准确率和查询检索方面均表现优异,为机器人部署提供了鲁棒、实时的解决方案。完整源代码、数据生成与评估流程将公开于https://github.com/DFKI-NI/DISC。
原文摘要 · Abstract (English)
Open-set semantic mapping enables language-driven robotic perception, but current instance-centric approaches are bottlenecked by context-depriving and computationally expensive crop-based feature extraction. To overcome this fundamental limitation, we introduce DISC (Dense Integrated Semantic Context), featuring a novel single-pass, distance-weighted extraction mechanism. By deriving high-fidelity CLIP embeddings directly from the vision transformer's intermediate layers, our approach eliminates the latency and domain-shift artifacts of traditional image cropping, yielding pure, mask-aligned semantic representations. To fully leverage these features in large-scale continuous mapping, DISC is built upon a fully GPU-accelerated architecture that replaces periodic offline processing with precise, on-the-fly voxel-level instance refinement. We evaluate our approach on standard benchmarks (Replica, ScanNet) and a newly generated large-scale-mapping dataset based on Habitat-Matterport 3D (HM3DSEM) to assess scalability across complex scenes in multi-story buildings. Extensive evaluations demonstrate that DISC significantly surpasses current state-of-the-art zero-shot methods in both semantic accuracy and query retrieval, providing a robust, real-time capable framework for robotic deployment. The full source code, data generation and evaluation pipelines will be made available at https://github.com/DFKI-NI/DISC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。