用大模型模拟人类寻物时的思维过程,提升智能体导航成功率。
CogNav: Cognitive Process Modeling for Object Goal Navigation with LLMs
- 构建基于大模型的动态认知状态机,模拟人类寻物时的思维变化。
- 在多个基准上实现至少14%的成功率提升,超越现有方法。
- 适合研究具身智能认知建模与机器人导航的开发者参考。
物体目标导航(ObjectNav)是具身人工智能中的基础任务,要求智能体在未见过的环境中定位目标物体。该任务极具挑战性,因为它同时依赖感知与认知能力,包括物体识别和决策判断。尽管视觉基础模型推动了感知性能的显著进步,但认知层面的进展仍受限于隐式学习或预设启发式规则。受神经科学发现启发——人类在陌生环境中搜索物体时会维持并动态更新精细认知状态,我们提出CogNav框架,利用大语言模型模拟这一认知过程。具体而言,采用有限状态机建模精细认知状态(从探索到识别),状态转移由大语言模型根据动态构建的异构认知地图决定,该地图包含场景的空间与语义信息。在HM3D、MP3D和RoboTHOR基准上的大量实验表明,该认知过程建模使物体导航成功率相对现有最优方法至少提升14%。
原文摘要 · Abstract (English)
Object goal navigation (ObjectNav) is a fundamental task in embodied AI, requiring an agent to locate a target object in previously unseen environments. This task is particularly challenging because it requires both perceptual and cognitive processes, including object recognition and decision-making. While substantial advancements in perception have been driven by the rapid development of visual foundation models, progress on the cognitive aspect remains constrained, primarily limited to either implicit learning through simulator rollouts or explicit reliance on predefined heuristic rules. Inspired by neuroscientific findings demonstrating that humans maintain and dynamically update fine-grained cognitive states during object search tasks in novel environments, we propose CogNav, a framework designed to mimic this cognitive process using large language models. Specifically, we model the cognitive process using a finite state machine comprising fine-grained cognitive states, ranging from exploration to identification. Transitions between states are determined by a large language model based on a dynamically constructed heterogeneous cognitive map, which contains spatial and semantic information about the scene being explored. Extensive evaluations on the HM3D, MP3D, and RoboTHOR benchmarks demonstrate that our cognitive process modeling significantly improves the success rate of ObjectNav at least by relative 14% over the state-of-the-arts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。