让视觉语言模型先说理再操作，提升机器人抓取定位精度。
TRACE: Textual Reasoning for Affordance Coordinate Extraction
- 引入文本推理链，让模型先解释空间操作逻辑
- 在Where2Place基准上达48.1%准确率，提升9.6%
- 推理数据越多效果越好，且注意力可解释
视觉语言模型（VLM）难以将高层指令转化为机器人操作所需的精确空间位置。现有视觉思维链方法计算开销大。本文提出TRACE（文本推理用于功能坐标提取），将文本思维链（CoR）融入功能预测流程。构建了由自主流水线生成的TRACE数据集，包含指令与明确的文本推理。微调VLM后，模型学会在执行前外部化空间推理过程。实验显示，经过TRACE微调的模型在主要的Where2Place（W2P）基准上达到48.1%准确率（相对提升9.6%），在更具挑战性的W2P(h)子集上达55.0%。消融实验表明性能随推理数据量增加而提升，验证了文本思维链的有效性。注意力分析显示模型关注点随推理步骤动态变化，具有可解释性。该工作证明训练VLM生成文本思维链是提升基于VLM的机器人控制精度、可靠性和可解释性的有效策略。数据集与代码已开源。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) struggle to translate high-level instructions into the precise spatial affordances required for robotic manipulation. While visual Chain-of-Thought (CoT) methods exist, they are often computationally intensive. In this work, we introduce TRACE (Textual Reasoning for Affordance Coordinate Extraction), a novel methodology that integrates a textual Chain of Reasoning (CoR) into the affordance prediction process. We use this methodology to create the TRACE dataset, a large-scale collection created via an autonomous pipeline that pairs instructions with explicit textual rationales. By fine-tuning a VLM on this data, our model learns to externalize its spatial reasoning before acting. Our experiments show that our TRACE-tuned model achieves state-of-the-art performance, reaching 48.1% accuracy on the primary Where2Place (W2P) benchmark (a 9.6% relative improvement) and 55.0% on the more challenging W2P(h) subset. Crucially, an ablation study demonstrates that performance scales directly with the amount of reasoning data used, confirming the CoR's effectiveness. Furthermore, analysis of the model's attention maps reveals an interpretable reasoning process where focus shifts dynamically across reasoning steps. This work shows that training VLMs to generate a textual CoR is an effective and robust strategy for enhancing the precision, reliability, and interpretability of VLM-based robot control. Our dataset and code are available at https://github.com/jink-ucla/TRACE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。