教小模型精准定位可交互区域,提升机器人视觉理解能力。
SpatialAfford: Teaching Compact VLMs Where to Look and Where to Ground for Affordance

- 先对齐注意力到真实交互区域,再优化坐标预测。
- 40亿参数模型在多个数据集上超越70亿以上基线。
- 适合资源受限的机器人场景,提升小模型感知精度。
可交互性定位旨在识别可操作区域(如抓握把手或按压按钮),而非整个物体,因此比通用视觉定位更具挑战性,尤其对用于具身智能的紧凑型视觉-语言模型(VLM)而言。现有序列级监督和强化学习虽提升了坐标预测质量,但紧凑的自回归VLM在生成坐标前仍缺乏可靠的可交互性感知注意力:模型虽能生成更优坐标标记,其跨模态注意力却仍分散且弱关联于真实可交互证据。为此,我们提出SpatialAfford,一种两阶段框架:首先通过空间注意力对齐(SAA)将注意力聚焦于真实可交互区域,随后使用空间感知的GRPO(Spatial-Aware GRPO)优化坐标预测。通过显式教导模型‘先看哪里’再‘准确定位’,SpatialAfford将可交互性定位从纯输出约束目标转变为以注意力为基础的空间推理。在ShareRobot-Bench、ReasonAff和PartAfford三个数据集上,SpatialAfford持续提升性能,其中40亿参数模型表现优于更强的70亿以上基线。
原文摘要 · Abstract (English)
Affordance grounding aims to localize the functional region for interaction, such as the handle to grasp or the button to press, rather than the whole object. This makes it more challenging than generic visual grounding because the target region is smaller, more ambiguous, and more dependent on task context, especially for compact vision-language models (VLMs) used in embodied settings. Recent sequence-level supervision and reinforcement learning improve coordinate prediction quality, yet compact autoregressive VLMs still lack reliable affordance-aware visual focus before coordinate generation: the model can produce better coordinate tokens while its cross-modal attention remains diffuse and weakly anchored to the true affordance evidence. To address it, we propose SpatialAfford, a two-stage framework that first aligns attention to the ground-truth affordance region through Spatial Attention Alignment (SAA), then refines coordinate prediction with Spatial-Aware GRPO. By explicitly teaching the model where to look before optimizing where to ground, SpatialAfford turns affordance grounding from a purely output-constrained objective into attention-grounded spatial reasoning. Across ShareRobot-Bench, ReasonAff, and PartAfford, SpatialAfford consistently improves affordance grounding, with a compact 4B model outperforming stronger 7B+ baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。