arXiv:2606.23717eess.IV2026-06

首次为多帧空间推理提供理论框架,揭示模型表现的关键制约因素。

SpaCE: Rethinking Spatial Capacity and Generalization in Multi-Frame Multimodal Large Language Models

论文配图:SpaCE: Rethinking Spatial Capacity and Generalization in Multi-Frame Multimodal Large Language Models
图 1 · 摘自论文原文
  • 从信息论出发,建立空间推理准确率的上限
  • 推导出样本复杂度与有效空间维度、帧数的关系
  • 证明显式3D表示与隐式推理的优劣切换条件

多模态大语言模型(MLLMs)在空间理解任务上取得显著进展,但其多帧空间推理的理论基础仍为空白。本文提出SpaCE,一个严谨的理论框架,用于刻画多帧输入下MLLMs的空间推理能力、样本复杂度和泛化保证。我们建立四项核心结果:首先,基于多帧观测与空间目标间的互信息,证明了空间推理准确率的信息论上限;其次,推导出样本复杂度为Θ(d_eff · K_max / (ε² · δ)),其中d_eff为有效空间维度,K_max约束学习后验的KL散度;第三,给出分布偏移下多帧空间推理的PAC-Bayes泛化界;第四,形式化刻画显式3D表示与隐式推理之间的偏差-方差权衡,识别出两者最优切换的条件。我们在MultiSPA、CA-VQA和SpatialRGPT基准上验证理论预测,表明边界具有良好的经验紧性,且帧间互补性是多帧空间能力的关键驱动力。该框架首次为理解多帧空间推理何时、为何、如何成功提供了原理性基础。

原文摘要 · Abstract (English)

Multi-modal large language models (MLLMs) have achieved remarkable empirical progress in spatial understanding through large-scale training on spatial visual question answering datasets. However, the theoretical foundations of multi-frame spatial reasoning remain entirely unexplored. We present SpaCE, a rigorous theoretical framework that characterizes the spatial reasoning capacity, sample complexity, and generalization guarantees of MLLMs operating on multi-frame inputs. We establish four main results. First, we prove an information-theoretic upper bound on spatial reasoning accuracy in terms of the mutual information between multi-frame observations and spatial targets. Second, we derive a sample complexity bound of order $Θ(d_{\mathrm{eff}} \cdot K_{\max} / (\varepsilon^2 \cdot δ))$, where $d_{\mathrm{eff}}$ is the effective spatial dimension and $K_{\max}$ bounds the KL divergence of the learned posterior. Third, we provide a PAC-Bayes generalization bound for multi-frame spatial reasoning under distribution shift. Fourth, we formally characterize the bias-variance trade-off between explicit 3D representations and implicit reasoning approaches, identifying the crossover conditions under which each paradigm is provably preferable. We validate our theoretical predictions on the MultiSPA, CA-VQA, and SpatialRGPT benchmarks, demonstrating that our bounds are empirically tight and that frame complementarity is the key driver of multi-frame spatial capacity. Our framework provides the first principled theoretical foundation for understanding when, why, and how multi-frame spatial reasoning in MLLs succeeds.

多帧推理理论分析空间理解大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。