让机器人在陌生环境里高效找多个物体，还能应对遮挡和混乱。
OVAMOS: A Framework for Open-Vocabulary Multi-Object Search in Unknown Environments
- 用视觉语言模型推断物体与环境关系，提升搜索效率
- 在120个模拟场景和真实办公室中成功率超基线，平均耗时减少37%
- 适合需要自主探索与容错的机器人导航任务
物体搜索是机器人在室内环境中执行的基础任务，但因观测不稳定而面临挑战，尤其是开放词汇模型。尽管基础模型（如大语言模型/视觉语言模型）可实现无直接可见情况下的推理，但从失败中恢复与重规划能力仍至关重要。多物体搜索（MOS）问题进一步增加复杂性，需同时追踪多个物体并充分探索新环境，观测不确定性成为主要障碍。为此，我们提出一个融合视觉语言模型推理、基于前沿的探索和部分可观测马尔可夫决策过程（POMDP）框架的解决方案，用于在新环境中解决MOS问题。视觉语言模型通过推断物体-环境关系提升搜索效率，基于前沿的探索引导未知空间导航，而POMDP建模观测不确定性，使系统可在遮挡和杂乱环境中实现故障恢复。我们在多个Habitat-Matterport3D（HM3D）场景中评估了120个模拟场景，并在50平方米的现实办公室中进行机器人实验，结果表明该框架在效率和成功率上均显著优于基线方法。
原文摘要 · Abstract (English)
Object search is a fundamental task for robots deployed in indoor building environments, yet challenges arise due to observation instability, especially for open-vocabulary models. While foundation models (LLMs/VLMs) enable reasoning about object locations even without direct visibility, the ability to recover from failures and replan remains crucial. The Multi-Object Search (MOS) problem further increases complexity, requiring the tracking multiple objects and thorough exploration in novel environments, making observation uncertainty a significant obstacle. To address these challenges, we propose a framework integrating VLM-based reasoning, frontier-based exploration, and a Partially Observable Markov Decision Process (POMDP) framework to solve the MOS problem in novel environments. VLM enhances search efficiency by inferring object-environment relationships, frontier-based exploration guides navigation in unknown spaces, and POMDP models observation uncertainty, allowing recovery from failures in occlusion and cluttered environments. We evaluate our framework on 120 simulated scenarios across several Habitat-Matterport3D (HM3D) scenes and a real-world robot experiment in a 50-square-meter office, demonstrating significant improvements in both efficiency and success rate over baseline methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。