教大模型先用专业视觉工具，再学会自己看懂细节
SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them
- 分三阶段训练：调用专业模型增强感知，强化工具使用，最后内化视觉能力
- 在MindCube上将模型准确率从29.3%提升至84.6%，内化后仍保持73.8%
- 适合需要精准空间理解的智能体任务，如机器人导航与交互
视觉语言模型（VLM）常用于具身智能体中解析视觉输入、推理空间关系并做出任务决策。然而存在根本性能力差距：通用VLM能理解整体任务但忽略决定成败的视觉细节；专用视觉模型虽能捕捉细节却无法转化为任务级判断。本文提出SpatialCLI框架，让VLM先借助专业视觉模型作为空间工具进行推理，再逐步内化其感知能力。该框架包含三个阶段：(1) Call阶段将专用视觉模型作为空间工具增强感知；(2) Learn阶段通过冷启动监督微调和代理式强化学习优化工具使用；(3) Internalize阶段将成功工具使用轨迹转化为内部知识。我们还构建了SpatialCLI-Bench，一个包含516个样本的基准测试，涵盖定位、分割、深度与姿态等组合感知任务。在MindCube上，SpatialCLI使Qwen3-VL-8B-Instruct准确率从29.3%提升至84.6%（使用工具），超越GPT-5.6 Sol（72.1%），内化后仍保持73.8%的性能。
原文摘要 · Abstract (English)
Vision-language models (VLMs) are increasingly used in embodied agents to interpret visual inputs, reason about spatial relationships, and make task-level decisions based on that reasoning. However, a fundamental capability mismatch remains: general VLMs can reason about the overall task but often miss the visual details that determine success, while specialist vision models can capture those details but cannot translate them into task-level decisions. In this work, we propose SpatialCLI, a framework that teaches VLMs to reason with spatial tools and progressively internalize the specialist perceptual capabilities they provide. SpatialCLI proceeds in three stages: (1) Call exposes specialist vision models as spatial tools to augment the VLM's perception; (2) Learn uses Cold-Start SFT and agentic RL to improve tool use; and (3) Internalize verbalizes successful tool-use trajectories to internalize specialist perceptual capabilities. We further introduce SpatialCLI-Bench, a 516-example benchmark for compositional perception across localization, segmentation, depth, and pose. On MindCube, SpatialCLI raises Qwen3-VL-8B-Instruct from 29.3% to 84.6% with tools, surpassing GPT-5.6 Sol with tools (72.1%), while retaining 73.8% without tools after internalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。