用大模型生成可读可编辑的语义地图，让机器人在室内规划更智能。
SENT Map -- Semantically Enhanced Topological Maps with Foundation Models
- 用视觉-大模型联合建图，生成可理解的语义地图
- 小模型在语义地图上规划成功率达90%以上
- 适合需理解场景的机器人导航与操作任务
我们提出SENT-Map，一种基于基础模型（FMs）的语义增强拓扑地图，用于表示室内环境，支持自主导航与操作。通过将环境以JSON文本格式表示，实现人类与大模型均可理解的语义信息添加与编辑，并在规划中将机器人锚定至已有节点，避免部署时出现不可行状态。该框架采用两阶段方法：首先通过视觉-大模型与操作员协同映射环境；随后利用SENT-Map表示与自然语言查询，在大模型中完成规划。实验表明，语义增强使小型本地部署的大模型也能成功规划室内环境。
原文摘要 · Abstract (English)
We introduce SENT-Map, a semantically enhanced topological map for representing indoor environments, designed to support autonomous navigation and manipulation by leveraging advancements in foundational models (FMs). Through representing the environment in a JSON text format, we enable semantic information to be added and edited in a format that both humans and FMs understand, while grounding the robot to existing nodes during planning to avoid infeasible states during deployment. Our proposed framework employs a two stage approach, first mapping the environment alongside an operator with a Vision-FM, then using the SENT-Map representation alongside a natural-language query within an FM for planning. Our experimental results show that semantic-enhancement enables even small locally-deployable FMs to successfully plan over indoor environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。