arXiv:2605.02130cs.CV2026-05被引 1

评测多模态大模型从空间位置到功能理解的高级推理能力

From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs

论文配图:From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs
图 1 · 摘自论文原文
  • 构建视频基准测试,评估模型的空间布局与功能推断能力
  • 1500+专家标注问题,涵盖多跳推理与情境化功能匹配
  • 揭示当前模型在空间记忆与功能推理融合上的显著短板

人类级代理智能不仅限于低层次几何感知,更在于从识别物体位置转向理解其用途。现有基准多聚焦于多模态大语言模型(MLLMs)的几何感知能力,难以评估实现具身智能所需的高阶认知。为此,我们提出空间-功能智能基准(SFI-Bench),基于超过1500个专家标注的问题,源自多样化的第一人称室内视频扫描。该基准系统评估两个互补维度的高级推理:(1) 结构化空间推理,要求理解复杂布局并形成连贯的空间表征;(2) 功能推理,涉及推断物体功能及其上下文依赖性效用。任务包括条件计数、多跳关系推理、功能配对和知识驱动故障排查,直接挑战模型整合感知、记忆与推理的能力。实验表明,当前MLLMs在结合空间记忆与功能推理及外部知识方面持续表现不佳,凸显实现具身智能的关键瓶颈。SFI-Bench因此成为衡量迈向更具认知能力与真正具身的多模态智能体进展的诊断工具。

原文摘要 · Abstract (English)

Human-level agentic intelligence extends beyond low-level geometric perception, evolving from recognizing where things are to understanding what they are for. While existing benchmarks effectively evaluate the geometric perception capabilities of multimodal large language models (MLLMs), they fall short of probing the higher-order cognitive abilities required for grounded intelligence. To address this gap, we introduce the Spatial-Functional Intelligence Benchmark (SFI-Bench), a video-based benchmark with over 1,500 expert-annotated questions derived from diverse egocentric indoor video scans. SFI-Bench systematically evaluates two complementary dimensions of advanced reasoning: (1) Structured Spatial Reasoning, which requires understanding complex layouts and forming coherent spatial representations, and (2) Functional Reasoning, which involves inferring object affordances and their context-dependent utility. The benchmark includes tasks such as conditional counting, multi-hop relational reasoning, functional pairing, and knowledge-grounded troubleshooting, directly challenging models to integrate perception, memory, and inference. Our experiments reveal that current MLLMs consistently struggle to combine spatial memory with functional reasoning and external knowledge, highlighting a critical bottleneck in achieving grounded intelligence. SFI-Bench therefore provides a diagnostic tool for measuring progress toward more cognitively capable and truly grounded multimodal agents.

多模态空间推理功能理解智能评测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。