提出30亿参数视觉语言动作模型，实现更自然的物体导航与可解释决策。
VISOR: VIsual Spatial Object Reasoning for Language-driven Object Navigation
- 通过三阶段推理框架，直接回答‘这是目标物体吗’和‘为何采取该动作’。
- 在AI2-THOR上导航成功率提升至87.3%，显著优于现有方法。
- 无需拼接多模型流程，兼具可解释性、泛化性与高效性，适合智能机器人研究者。
语言驱动的物体导航要求智能体理解自然语言描述的目标物体，这些描述结合了内在属性与外在空间关系以实现实例识别与常识性导航。现有方法要么采用端到端训练的视觉-语言嵌入模型，难以泛化且缺乏动作级可解释性；要么依赖基于大语言模型（LLMs）与开集物体检测器的模块化零样本流水线，存在误差传播、计算成本高且难以将推理结果反馈至导航策略的问题。为此，我们提出一个紧凑的30亿参数视觉-语言-动作（VLA）智能体，实现类人具身推理，同时完成物体识别与动作选择，无需拼接多模型流水线。不同于原始嵌入匹配，本模型采用显式图像引导的推理过程，直接回答‘这是目标物体吗？’和‘我为何要执行此动作？’。推理分三个阶段展开：‘思考’、‘思考摘要’与‘行动’，从而提升可解释性、增强泛化能力并实现更高效的导航。代码与数据集将在论文接受后公开。
原文摘要 · Abstract (English)
Language-driven object navigation requires agents to interpret natural language descriptions of target objects, which combine intrinsic and extrinsic attributes for instance recognition and commonsense navigation. Existing methods either (i) use end-to-end trained models with vision-language embeddings, which struggle to generalize beyond training data and lack action-level explainability, or (ii) rely on modular zero-shot pipelines with large language models (LLMs) and open-set object detectors, which suffer from error propagation, high computational cost, and difficulty integrating their reasoning back into the navigation policy. To this end, we propose a compact 3B-parameter Vision-Language-Action (VLA) agent that performs human-like embodied reasoning for both object recognition and action selection, removing the need for stitched multi-model pipelines. Instead of raw embedding matching, our agent employs explicit image-grounded reasoning to directly answer "Is this the target object?" and "Why should I take this action?" The reasoning process unfolds in three stages: "think", "think summary", and "action", yielding improved explainability, stronger generalization, and more efficient navigation. Code and dataset available upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。