高分解谜不代表真理解空间，模型靠枚举文本路径骗分。
From Pixels to BFS: High Maze Accuracy Does Not Imply Visual Planning
- 把图像转成文字网格后暴力搜索路径，像在文字里广度优先遍历
- 16个模型平均只靠纯推理做对2-12%，超大迷宫直接超限失败
- 适合研究模型是否真会空间规划的人看
多模态模型解视觉空间题是真规划，还是在文本中暴力搜？我们构建了包含110个程序生成迷宫图像的基准测试集 extsc{MazeBench}，涵盖九组受控条件，评估了来自OpenAI、Anthropic、Google和Alibaba的16种模型配置。GPT-5.4 解对91%，Gemini 3.1 Pro 解对79%，但这些分数具有误导性：模型通常将图像转化为文字网格，再逐步枚举路径，每题消耗1,710–22,818个标记，而人类可快速完成。未增加推理预算时，所有配置仅能正确2–12%；在20×20的超难迷宫中，全部因达到标记上限而失败。定性分析揭示一种通用两阶段策略：图像到网格转换，随后在标记层面进行搜索，实质为文字中的广度优先搜索（BFS）。文字网格消融实验显示，当给定正确网格时，Claude Sonnet 4.6准确率从6%跃升至80%，说明弱于视觉提取，强于下游搜索。即使明确要求不构造网格或做图搜索，模型仍回归同种枚举策略。因此， extsc{MazeBench} 表明，视觉规划任务的高准确率并不意味着具备类人空间理解能力。
原文摘要 · Abstract (English)
How do multimodal models solve visual spatial tasks -- through genuine planning, or through brute-force search in token space? We introduce \textsc{MazeBench}, a benchmark of 110 procedurally generated maze images across nine controlled groups, and evaluate 16 model configurations from OpenAI, Anthropic, Google, and Alibaba. GPT-5.4 solves 91\% and Gemini 3.1 Pro 79\%, but these scores are misleading: models typically translate images into text grids and then enumerate paths step by step, consuming 1,710--22,818 tokens per solve for a task humans do quickly. Without added reasoning budgets, all configurations score only 2--12\%; on 20$\times$20 ultra-hard mazes, they hit token limits and fail. Qualitative traces reveal a common two-stage strategy: image-to-grid translation followed by token-level search, effectively BFS in prose. A text-grid ablation shows Claude Sonnet 4.6 rising from 6\% on images to 80\% when given the correct grid, isolating weak visual extraction from downstream search. When explicitly instructed not to construct a grid or perform graph search, models still revert to the same enumeration strategy. \textsc{MazeBench} therefore shows that high accuracy on visual planning tasks does not imply human-like spatial understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。