arXiv:2510.07181cs.ROcs.AI2025-10被引 30

让视觉语言模型用工具做精准几何计算,实现机器人操作的厘米级精度。

TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Robotics

  • 通过调用外部工具生成代码,让模型执行精确几何运算。
  • 在真实机器人任务中实现厘米级定位精度,超越现有方法。
  • 适合需要高精度空间推理的机器人、自动驾驶场景。

视觉语言模型(VLMs)在空间推理方面表现优异,但本质上仅具备定性精度,缺乏实际机器人应用所需的计算精度。现有方法未能利用深度传感器和相机标定提供的度量信息,将几何问题简化为模式识别任务,无法实现厘米级准确度。本文提出TIGeR(Tool-Integrated Geometric Reasoning),一种新框架,使VLMs从感知估测器转变为几何计算机,通过外部工具生成并执行精确几何计算。TIGeR不试图将复杂几何操作内化到神经网络中,而是让模型识别几何推理需求,生成相应计算代码,并调用专用库进行精确求解。为支持该范式,我们构建了TIGeR-300K数据集,涵盖点变换、位姿估计和空间兼容性验证,包含工具调用序列与中间计算过程。采用两阶段训练流程(监督微调+SFT与强化微调+RFT),结合层级奖励设计,TIGeR在几何推理基准上达到当前最优性能,并在真实机器人操作任务中实现厘米级精度。

原文摘要 · Abstract (English)

Vision-Language Models (VLMs) have shown remarkable capabilities in spatial reasoning, yet they remain fundamentally limited to qualitative precision and lack the computational precision required for real-world robotics. Current approaches fail to leverage metric cues from depth sensors and camera calibration, instead reducing geometric problems to pattern recognition tasks that cannot deliver the centimeter-level accuracy essential for robotic manipulation. We present TIGeR (Tool-Integrated Geometric Reasoning), a novel framework that transforms VLMs from perceptual estimators to geometric computers by enabling them to generate and execute precise geometric computations through external tools. Rather than attempting to internalize complex geometric operations within neural networks, TIGeR empowers models to recognize geometric reasoning requirements, synthesize appropriate computational code, and invoke specialized libraries for exact calculations. To support this paradigm, we introduce TIGeR-300K, a comprehensive tool-invocation-oriented dataset covering point transformations, pose estimation, and spatial compatibility verification, complete with tool invocation sequences and intermediate computations. Through a two-stage training pipeline combining supervised fine-tuning (SFT) and reinforcement fine-tuning (RFT) with our proposed hierarchical reward design, TIGeR achieves SOTA performance on geometric reasoning benchmarks while demonstrating centimeter-level precision in real-world robotic manipulation tasks.

机器人几何推理视觉语言模型工具调用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。