评测视觉语言模型的空间逻辑推理能力，发现顶尖模型仍表现不佳。
SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models
- 构建包含9605个问答对的实地室内场景基准
- 41个主流模型在复杂空间推理任务中表现普遍较差
- 提出递归场景图辅助推理，显著提升模型能力
视觉语言模型（VLMs）因其出色的理解与推理能力被广泛应用于现实场景。尽管在常规视觉问答和逻辑推理方面表现优异，它们在复杂真实环境中的合理决策能力仍显不足。本文定义此能力为「空间逻辑推理」，不仅要求理解复杂场景中物体间的空间关系，还需把握多步骤任务中的逻辑依赖。为此，我们提出SpatiaLQA基准，涵盖241个真实室内场景生成的9,605个问答对。对41个主流VLM进行实验发现，即使最先进的模型也难以胜任空间逻辑推理。为此，我们提出递归场景图辅助推理方法，利用视觉基础模型逐步分解复杂场景为任务相关场景图，有效提升模型空间逻辑推理能力，优于所有现有方法。代码与数据集已开源。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) have been increasingly applied in real-world scenarios due to their outstanding understanding and reasoning capabilities. Although VLMs have already demonstrated impressive capabilities in common visual question answering and logical reasoning, they still lack the ability to make reasonable decisions in complex real-world environments. We define this ability as spatial logical reasoning, which not only requires understanding the spatial relationships among objects in complex scenes, but also the logical dependencies between steps in multi-step tasks. To bridge this gap, we introduce Spatial Logical Question Answering (SpatiaLQA), a benchmark designed to evaluate the spatial logical reasoning capabilities of VLMs. SpatiaLQA consists of 9,605 question answer pairs derived from 241 real-world indoor scenes. We conduct extensive experiments on 41 mainstream VLMs, and the results show that even the most advanced models still struggle with spatial logical reasoning. To address this issue, we propose a method called recursive scene graph assisted reasoning, which leverages visual foundation models to progressively decompose complex scenes into task-relevant scene graphs, thereby enhancing the spatial logical reasoning ability of VLMs, outperforming all previous methods. Code and dataset are available at https://github.com/xieyc99/SpatiaLQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。