让大模型用可执行的矢量图思考，提升空间推理能力
MentalThink: Shaping Thoughts in Mental SVG World
- 用SVG代码作为中间表示，实现可视化的多轮推理
- 在VSIBench上达55.1%，MindCube上达76.0%准确率
- 适合需要空间想象与视觉推理的任务场景
我们提出MentalThink，一种视觉-符号推理范式，为多模态大模型（MLLMs）赋予“心智可视化”能力。核心是基于SVG的思考流程：模型学习生成、渲染并解释可缩放矢量图形（SVG）代码，作为多轮推理的中间视觉表征。通过创建结构化矢量草图，模型能外化空间假设，经确定性渲染进行检查，并在受限几何空间中推理，模拟人类心智意象过程。通过两阶段训练框架实现：先用监督微调（SFT）对齐SVG语法，再用多轮强化学习（RL）激励对中间视觉假设的迭代检验、修正与优化。大量评估表明，MentalThink在空间理解与推理基准测试中表现优异，如在VSIBench上达到55.1%，在MindCube上达76.0%。结果证明，可执行的矢量图形为动态视角转换、视觉反思与组合场景构建提供了可验证的视觉工作区。
原文摘要 · Abstract (English)
We introduce MentalThink, a visual-symbolic reasoning paradigm that equips Multimodal LLMs (MLLMs) with an executable mechanism for "mental" visualization. The core of MentalThink is a think-with-SVG pipeline, where the model learns to generate, render, and interpret scalable vector graphics (SVG) code as an intermediate visual representation for multi-turn reasoning. By creating structured vector sketches, the model can externalize spatial hypotheses, inspect them through deterministic rendering, and reason within a constrained geometric space, effectively mimicking the human process of mental imagery. We instantiate this paradigm through a two-stage training framework, combining Supervised Fine-Tuning (SFT) for SVG syntactic alignment with multi-turn Reinforcement Learning (RL) to encourage iterative inspection, revision, and refinement of intermediate visual hypotheses. Extensive evaluations demonstrate that MentalThink achieves superior performance on spatial understanding and reasoning benchmarks (e.g., 55.1% on VSIBench, 76.0% on MindCube), showing that executable vector graphics provide a verifiable visual workspace for dynamic perspective taking, visual reflection, and compositional scene construction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。