arXiv:2603.18662cs.AI2026-03

让AI学会主动画图解几何题,提升推理能力

Thinking with Constructions: A Benchmark and Policy Optimization for Visual-Text Interleaved Geometric Reasoning

  • 提出视觉与文本交替的思维链框架,动态生成辅助图形
  • 在4334道几何题上实现3.51%性能提升,优于现有模型
  • 适合研究多模态推理、AI数学解题的学者和开发者

几何推理本质是'通过作图思考'——动态操作视觉辅助工具以连接问题条件与解法。然而现有多模态大模型仅能被动分析静态图示,缺乏何时、如何构建有效视觉辅助的战略知识。为此,我们提出视觉-文本交错的思维链框架,并构建首个基准数据集GeoAux-Bench,包含4,334道几何题,其文本构造步骤与真实视觉更新对齐。初步研究发现:(1) 视觉与文本交替的辅助方式优于单一模态,能无损捕捉几何协同效应;(2) 正确作图显著降低推理困惑度,具有熵减作用。基于此,我们提出行动适用性策略优化(A2PO),一种强化学习范式,通过反事实采样区分必要与冗余构造,实现自适应奖励调节。实验表明,该方法使多模态大模型可选择性使用辅助作图,相较强基线提升3.51%。代码与数据已开源。

原文摘要 · Abstract (English)

Geometric reasoning inherently requires "thinking with constructions" -- the dynamic manipulation of visual aids to bridge the gap between problem conditions and solutions. However, existing Multimodal Large Language Models (MLLMs) are largely confined to passive inference with static diagrams, lacking the strategic knowledge of when and how to construct effective visual aids. To address this, we present a framework for Visual-Text Interleaved Chain-of-Thought. We first introduce GeoAux-Bench, the first benchmark comprising 4,334 geometry problems that aligns textual construction steps with ground-truth visual updates. Our pilot study reveals two critical insights: (1) interleaved visual-textual aids outperform single-modality counterparts, which cannot losslessly capture geometric synergy; and (2) valid constructions act as entropy reducers, strongly correlating with reduced reasoning perplexity. Building on these findings, we propose Action Applicability Policy Optimization (A2PO), a reinforcement learning paradigm for mastering strategic construction. A2PO employs Adaptive Reward Shaping to regulate the timing and quality of visual aids via counterfactual sampling to distinguish necessary from redundant constructions. Experiments demonstrate our approach enables MLLMs to leverage selective auxiliary constructions, yielding a 3.51% gain over strong baselines. Code and data are available on GitHub.

几何推理多模态强化学习思维链

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。