让AI推理时像钉住物体一样锁定视觉证据,避免思路跑偏。
RoboPIN: Grounded Embodied Reasoning via Pinned Chain-of-Thought

- 每步推理都绑定视觉锚点,确保对象定位和身份一致。
- 40亿参数模型在14项任务上超越70亿参数主流模型12%。
- 适合需要精准空间推理的机器人、导航系统开发者。
具身推理要求模型在物理环境中感知任务相关物体与空间,并在整个多步推理过程中保持视觉一致性。然而,当前视觉语言模型依赖纯文本或坐标增强的思维链,实体指代隐晦模糊,易导致推理脱离视觉证据、实体指代漂移、推理路径与最终答案因果脱节,尤其在多视角场景下因视角间外观变化而加剧。为此,我们提出结构化推理范式Pinned Chain-of-Thought(PinCoT),将每个推理步骤与视觉证据“钉住”:引入推理锚点,将每个任务相关实体绑定至包含实体名、唯一身份、视图索引和空间定位的结构化视觉锚点,实现跨步骤与跨视角的一致实体追踪。我们构建全自动数据生成流水线,创建高质量的PinCoT格式数据集PIN-170K。随后通过三阶段后训练构建RoboPIN,逐步注入具身知识、结构化推理能力与过程监督对齐,奖励机制直接约束推理中锚点定位与身份一致性。在涵盖具身空间推理、多视角推理与指代任务的14个基准上,仅40亿参数的RoboPIN平均超越70亿级开源具身模型,相较最强70亿基线Mimo-Embodied提升12%。进一步分析表明,PinCoT显著提升定位准确率与跨步身份一致性,验证了过程监督的有效性。
原文摘要 · Abstract (English)
Embodied reasoning requires models to perceive task-relevant objects and spaces in physical environments and maintain consistent visual grounding throughout multi-step reasoning. However, current vision-language models rely on text-only or coordinate-augmented chain-of-thought, where entity references remain implicit and ambiguous. This may cause the reasoning process to decouple from visual evidence, entity references to drift across steps, and a causal disconnection between the reasoning trajectory and the final answer, with these problems further amplified in multi-view scenarios due to cross-view appearance changes. To address these issues, we propose Pinned Chain-of-Thought (PinCoT), a structured reasoning paradigm that pins every reasoning step to visual evidence. PinCoT introduces the concept of reasoning anchor, which binds each task-relevant entity to a structured visual anchor with entity name, unique identity, view index, and spatial grounding, enabling consistent entity tracking across reasoning steps and views. We build a fully automated data generation pipeline to construct PIN-170K, a high-quality PinCoT-formatted reasoning dataset. We then train RoboPIN through three-stage post-training that progressively injects embodied knowledge, structured reasoning ability, and process-supervised alignment, with rewards that directly constrain both anchor localization and identity consistency during reasoning. On 14 benchmarks covering embodied spatial reasoning, multi-view reasoning, and pointing, RoboPIN with only 4B parameters surpasses 7B level open-source embodied models on average, achieving a 12% average improvement over the strongest 7B baseline, Mimo-Embodied. Further analysis shows that PinCoT improves grounding accuracy and cross-step identity consistency, validating the effectiveness of process supervision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。