让无人机按语言指令精准找物,同时不遗漏地图区域。
Semantic-Aware Guided Drone Exploration for Language-Conditioned 3D Indoor Mapping

- 用语义感知重排探索优先级,结合图像与语言模型识别目标。
- 在9组测试中比现有方法快13.7倍,物体发现率显著提升。
- 真实飞行验证有效,适合需精准定位的室内导航任务。
我们提出语义感知引导探索(SAGE),一种在未知三维室内环境中进行开放词汇探索的系统,可在保持覆盖率的同时利用语义线索重新排序前沿选择。基于FALCON体素探索器,SAGE通过四个核心组件集成对比语言-图像预训练模型(CLIP):以对象为中心的嵌入存储、将近期观测投影至自由-未知边界的时序缓存、高相似度检测的对象前沿,以及统一的语义-几何规划代价函数。该代价函数限制语义重加权的影响范围,确保前沿优先级不牺牲整体覆盖率。在Matterport3D仿真中,SAGE在多组地图-查询配对中优于FALCON和仅依赖语义的消融实验。相较于未知环境中的寻物(FTU),SAGE在九组共享地图-查询配对中探索速度提升9.0至25.9倍,平均提速13.7倍。此外,SAGE的体素吞吐量显著高于FTU。最后,我们在两个真实环境中的五次飞行中部署了SAGE,使用搭载本地传感与规划的Modal AI Starling 2四旋翼无人机,离线执行CLIP推理。对比SAGE与FALCON,尽管FALCON探索更快、路径更短,但SAGE在物体发现方面表现更优。
原文摘要 · Abstract (English)
We present Semantic-Aware Guided Exploration, SAGE, a system for open-vocabulary exploration in unknown 3D indoor environments that preserves coverage-oriented behavior while allowing semantic cues to reprioritize frontier selection. Building on the FALCON volumetric explorer, SAGE integrates Contrastive Language-Image Pre-training (CLIP) via four key components: object-centric embedding storage, a temporal cache that projects recent observations onto the free-unknown boundary, object frontiers for high-similarity detections, and a unified semantic-geometric planning cost. This cost function bounds semantic reweighting influence, ensuring frontiers are prioritized without sacrificing total coverage. In Matterport3D-based simulations, SAGE outperforms FALCON and a semantic-only ablation in object discovery across map-query pairs. Compared to Finding Things in the Unknown (FTU), SAGE completes exploration 9.0 to 25.9 times faster across the nine shared map-query pairs, achieving a mean speedup of 13.7. Furthermore, SAGE achieves substantially higher volumetric throughput than FTU. Finally, we deploy SAGE in five real-world flights in two environments on a Modal AI Starling 2 quadrotor with onboard sensing and planning, and offboard CLIP inference. Comparing SAGE and FALCON, we find that while FALCON results in faster exploration and shorter mapping trajectories, SAGE outperforms FALCON in terms of object discovery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。