让机器人看懂招牌并用地图导航,在陌生商场高效找店。
Signage-Aware Exploration in Open World using Venue Maps
- 用扩散模型+2D-3D融合识别复杂招牌文字。
- 结合地图引导探索,找店效率超越现有方法。
- 适合需要地图导航的室内机器人场景。
当前探索方法在未知开放世界中难以搜索商店或餐厅,因缺乏先验知识。人类可借助场所地图,将场景中的招牌文字与地图上的地标名称关联,辅助规划探索路径。但招牌文字形状样式多样、多视角不一致,导致机器人识别困难;且现实环境与地图间存在差异,制约文本信息融入规划。本文提出一种新型招牌感知探索系统,使机器人有效利用场所地图。我们设计了一种基于扩散的文本实例检索方法,结合2D到3D语义融合策略,精准检测与识别招牌文字。此外,构建了地图引导的探索-利用规划器,通过地图推导方向性启发式策略探索未知区域,并主动靠近调整姿态以提升识别效果。在大型购物中心的实验表明,本方法在招牌识别性能和搜索效率上均优于主流文本定位方法及传统探索策略。项目网站:https://sites.google.com/view/signage-aware-exploration。
原文摘要 · Abstract (English)
Current exploration methods struggle to search for shops or restaurants in unknown open-world environments due to the lack of prior knowledge. Humans can leverage venue maps that offer valuable scene priors to aid exploration planning by correlating the signage in the scene with landmark names on the map. However, arbitrary shapes and styles of the texts on signage, along with multi-view inconsistencies, pose significant challenges for robots to recognize them accurately. Additionally, discrepancies between real-world environments and venue maps hinder the integration of text-level information into the planners. This paper introduces a novel signage-aware exploration system to address these challenges, enabling the robots to utilize venue maps effectively. We propose a signage understanding method that accurately detects and recognizes the texts on signage using a diffusion-based text instance retrieval method combined with a 2D-to-3D semantic fusion strategy. Furthermore, we design a venue map-guided exploration-exploitation planner that balances exploration in unknown regions using directional heuristics derived from venue maps and exploitation to get close and adjust orientation for better recognition. Experiments in large-scale shopping malls demonstrate our method's superior signage recognition performance and search efficiency, surpassing state-of-the-art text spotting methods and traditional exploration approaches. Project website: https://sites.google.com/view/signage-aware-exploration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。