用数学教育理论分析大模型解题思路，发现思维路径越聚焦越准确。
SHAPE of Chain-of-Thought in Math Reasoning
- 从数学解释和解题策略两角度拆解模型推理过程
- 模型正确率与专注特定思维空间强相关，而非广泛探索
- 可指导训练优化，提升模型数学推理能力
大语言模型在数学推理任务上表现优异，但其内在数学思维机制仍不清晰。本文提出 exttt{SHAPE} 框架，基于数学教育中的两个视角分析思维链（CoT）轨迹：(1) 语义空间——模型对问题的数学理解类型（如代数、几何），(2) 启发式策略——在该空间中采取的具体操作（如简化问题、逆向推导）。分析显示，模型使用的启发式策略比传统 CoT 特征更能解释最终答案正确性；且正确解题往往集中于少数语义空间，类似人类行为。进一步发现，强化学习训练导致启发式使用趋于单一。通过鼓励多样化启发式进行后训练，显著提升了模型准确率。 exttt{SHAPE} 为理解与优化大模型数学推理提供了理论基础和新路径。代码已开源。
原文摘要 · Abstract (English)
Large language models (LLMs) achieve strong performance on mathematical reasoning benchmarks, yet the mathematically meaningful skills underlying their reasoning remain underexplored. We introduce \texttt{SHAPE}, a framework that analyzes Chain-of-Thought (CoT) trajectories through two lenses developed in mathematics education: (1) semantic spaces: the model's evolving mathematical interpretations of a problem (e.g., algebraic, geometric), and (2) heuristics: the specific mathematical actions taken within those spaces (e.g., simplifying the problem, working backward). We first use \texttt{SHAPE} to analyze the reasoning patterns of various models. Our findings reveal that the mathematical heuristics employed by a model better explain final answer correctness than traditional CoT features. Furthermore, models are likely to reach correct solutions by concentrating their reasoning effort within a few semantic spaces rather than exploring many disparate ones -- a pattern consistent with human behavior. Next, we utilize the \texttt{SHAPE} lens to evaluate whether post-training truly enhances mathematical proficiency. We find that reinforcement learning induces mode-seeking in heuristic usage. Lastly, we post-train LLMs by promoting diverse heuristics and demonstrate its effectiveness in improving accuracy. Overall, \texttt{SHAPE} provides a theoretically-grounded diagnostic framework for decoding LLM reasoning and offers a new path toward post-training LLMs for math reasoning. The code for our model is available at https://github.com/holi-lab/SHAPE-of-CoT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。