让大模型像人一样分步看图解数学题,显著提升推理能力。
ViRC: Enhancing Visual Interleaved Mathematical CoT with Reason Chunking
- 将解题过程拆分为关键推理单元,结合图文信息逐步推导
- 在多个数学基准上比基线模型平均提升18.8%准确率
- 适合研究多模态推理、数学AI或认知启发式模型的学者
思维链(CoT)虽显著提升了大语言模型的推理能力,但在多模态数学任务中仍面临挑战。现有多模态大模型通常仅从单一静态数学图像进行文本推理,忽略了人类在解题时动态观察图像并分步推导的过程。受认知科学中米勒定律启发,我们提出ViRC框架,引入推理切块机制,将多模态数学思维链划分为连续的关键推理单元(CRUs),模拟专家解题模式。每个CRU保证中间命题的内部文本一致性,同时跨单元整合视觉信息以生成后续命题,实现结构化推理。为此,我们构建了CRUX数据集,使用三种视觉工具和四种推理模式,为每道数学题标注多个推理路径中的显式CRUs。基于该数据集,我们设计渐进式训练策略,包含指令微调(Instructional SFT)、练习微调(Practice SFT)和策略强化学习(Strategic RL),以增强模型的推理切块能力。最终的ViRC-7B模型在多个数学基准上相较基线平均提升18.8%。代码已开源:https://github.com/Leon-LihongWang/ViRC。
原文摘要 · Abstract (English)
CoT has significantly enhanced the reasoning ability of LLMs while it faces challenges when extended to multimodal domains, particularly in mathematical tasks. Existing MLLMs typically perform textual reasoning solely from a single static mathematical image, overlooking dynamic visual acquisition during reasoning. In contrast, humans repeatedly examine visual image and employ step-by-step reasoning to prove intermediate propositions. This strategy of decomposing the problem-solving process into key logical nodes adheres to Miller's Law in cognitive science. Inspired by this insight, we propose a ViRC framework for multimodal mathematical tasks, introducing a Reason Chunking mechanism that structures multimodal mathematical CoT into consecutive Critical Reasoning Units (CRUs) to simulate human expert problem-solving patterns. CRUs ensure intra-unit textual coherence for intermediate proposition verification while integrating visual information across units to generate subsequent propositions and support structured reasoning. To this end, we present CRUX dataset by using three visual tools and four reasoning patterns to provide explicitly annotated CRUs across multiple reasoning paths for each mathematical problem. Leveraging the CRUX dataset, we propose a progressive training strategy inspired by human cognitive learning, which includes Instructional SFT, Practice SFT, and Strategic RL, aimed at further strengthening the Reason Chunking ability of the model. The resulting ViRC-7B model achieves a 18.8% average improvement over baselines across multiple mathematical benchmarks. Code is available at https://github.com/Leon-LihongWang/ViRC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。