arXiv:2512.20595cs.CLcs.AI2025-12被引 1

用魔方评测多模态大模型的空间推理能力,发现闭源模型明显领先。

Cube Bench: A Benchmark for Spatial Visual Reasoning in MLLMs

  • 设计魔方基准测试,拆解空间推理为五项技能。
  • 模型准确率随魔方打乱深度急剧下降,纠错能力普遍差。
  • 适合评估大模型的顺序决策与自我修正能力。

我们提出 Cube Bench,一个基于魔方的基准测试,用于评估多模态大语言模型(MLLMs)的空间与序列推理能力。该基准将性能分解为五项技能:(i) 从图像和文本重建魔方面;(ii) 选择最优下一步操作;(iii) 预测候选动作的结果而不执行;(iv) 执行多步计划并从错误中恢复;(v) 检测并修正自身错误。使用统一的打乱状态、相同提示和解析器,以及单一的求解距离指标,我们对比了七种近期 MLLMs 在不同打乱深度下的表现。结果显示,模型准确率随打乱深度显著下降;一旦轨迹停滞或偏离,模型极少能恢复;高面重建准确率并不保证动作选择或多步执行能力。闭源模型在单步感知和多步控制任务上均显著领先,而开源模型在最复杂场景下接近随机水平;即使最强模型在更高复杂度下也出现性能退化。通过反思式自我修正可带来小幅提升,但可能引发过度思考。Cube Bench 提供了一个紧凑、可复现的序列空间推理探针。

原文摘要 · Abstract (English)

We introduce Cube Bench, a Rubik's-cube benchmark for evaluating spatial and sequential reasoning in multimodal large language models (MLLMs). The benchmark decomposes performance into five skills: (i) reconstructing cube faces from images and text, (ii) choosing the optimal next move, (iii) predicting the outcome of a candidate move without applying it, (iv) executing multi-step plans while recovering from mistakes, and (v) detecting and revising one's own errors. Using a shared set of scrambled cube states, identical prompts and parsers, and a single distance-to-solved metric, we compare recent MLLMs side by side as a function of scramble depth. Across seven MLLMs, accuracy drops sharply with depth; once a trajectory stalls or diverges, models rarely recover, and high face-reconstruction accuracy does not guarantee competent action selection or multi-step execution. A pronounced closed- vs open-source gap emerges: the strongest closed model leads on both single-step perception tasks and multi-step control tasks, while open-weight models cluster near chance on the hardest settings; yet even the best MLLM degrades at higher cube complexity. A simple self-correction via reflective thinking yields modest gains but can also introduce overthinking. Cube Bench offers a compact, reproducible probe of sequential spatial reasoning in MLLMs.

多模态模型空间推理魔方基准自纠正

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。