arXiv:2511.15333cs.RO2025-11

用粗到精的两阶段方法,让AI更准地理解空间指令中的位置描述。

C2F-Space: Coarse-to-Fine Space Grounding for Spatial Instructions using Vision-Language Models

  • 先用视觉语言模型粗定位,再通过超像素细化区域
  • 在新基准上成功率和交并比均超越现有方法
  • 适合机器人导航与交互等需要精准空间理解的任务

空间定位指根据自然语言指令定位空间参照物。传统方法难以处理距离、几何关系等复杂推理,而视觉语言模型虽有强推理能力,却难以输出细粒度区域。为此,我们提出C2F-Space,一种粗到精的空间定位框架:首先利用基于网格的提示与提议-验证策略,由VLM估计出空间一致的粗略区域(如椭圆);随后通过超像素化对区域进行局部适应性细化,使其贴合周围环境而不过度松弛。我们构建了新的空间定位基准,对比五种先进基线,采用成功率与交并比评估。C2F-Space显著优于所有基线。消融实验验证了两阶段模块的有效性及其协同作用。最终,我们在模拟机器人抓取任务中展示了该方法的应用潜力。

原文摘要 · Abstract (English)

Space grounding refers to localizing a set of spatial references described in natural language instructions. Traditional methods often fail to account for complex reasoning -- such as distance, geometry, and inter-object relationships -- while vision-language models (VLMs), despite strong reasoning abilities, struggle to produce a fine-grained region of outputs. To overcome these limitations, we propose C2F-Space, a novel coarse-to-fine space-grounding framework that (i) estimates an approximated yet spatially consistent region using a VLM, then (ii) refines the region to align with the local environment through superpixelization. For the coarse estimation, we design a grid-based visual-grounding prompt with a propose-validate strategy, maximizing VLM's spatial understanding and yielding physically and semantically valid canonical region (i.e., ellipses). For the refinement, we locally adapt the region to surrounding environment without over-relaxed to free space. We construct a new space-grounding benchmark and compare C2F-Space with five state-of-the-art baselines using success rate and intersection-over-union. Our C2F-Space significantly outperforms all baselines. Our ablation study confirms the effectiveness of each module in the two-step process and their synergistic effect of the combined framework. We finally demonstrate the applicability of C2F-Space to simulated robotic pick-and-place tasks.

空间定位视觉语言模型机器人多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。