arXiv:2508.02583cs.AIcs.LG2025-08AAAI被引 3

用因果图增强大模型数学推理能力,提升解题准确性。

CAMA: Enhancing Mathematical Reasoning in Large Language Models with Causal Knowledge

  • 构建数学因果图MCG,显式编码知识点间依赖关系。
  • 在真实数据集上,数学题准确率显著提升,优于无结构引导方法。
  • 适合需要强逻辑推理的数学任务,尤其对复杂问题有效。

大型语言模型(LLMs)在多种任务中表现优异,但在复杂数学推理上仍存在挑战,根本原因在于深层结构依赖。为此,我们提出因果数学家(CAMA),一种两阶段因果框架,为LLMs引入显式可复用的数学结构。学习阶段,CAMA结合模型先验与因果发现算法,基于问题-解答对语料库构建数学因果图(MCG),编码关键知识点及其因果依赖。随后通过精选问题-解答对的迭代反馈进一步优化MCG。推理阶段,针对新问题,CAMA动态从MCG中提取任务相关子图,基于问题内容与模型中间推理轨迹进行条件化,将该子图注入LLM以指导推理。实验证明,CAMA在真实数据集上显著提升模型对复杂数学问题的性能;结构化引导优于无结构方式,且引入非对称因果关系带来的提升高于仅使用对称关联。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have demonstrated strong performance across a wide range of tasks, yet they still struggle with complex mathematical reasoning, a challenge fundamentally rooted in deep structural dependencies. To address this challenge, we propose \textbf{CA}usal \textbf{MA}thematician (\textbf{CAMA}), a two-stage causal framework that equips LLMs with explicit, reusable mathematical structure. In the learning stage, CAMA first constructs the \textbf{M}athematical \textbf{C}ausal \textbf{G}raph (\textbf{MCG}), a high-level representation of solution strategies, by combining LLM priors with causal discovery algorithms applied to a corpus of question-solution pairs. The resulting MCG encodes essential knowledge points and their causal dependencies. To better align the graph with downstream reasoning tasks, CAMA further refines the MCG through iterative feedback derived from a selected subset of the question-solution pairs. In the reasoning stage, given a new question, CAMA dynamically extracts a task-relevant subgraph from the MCG, conditioned on both the question content and the LLM's intermediate reasoning trace. This subgraph, which encodes the most pertinent knowledge points and their causal dependencies, is then injected back into the LLM to guide its reasoning process. Empirical results on real-world datasets show that CAMA significantly improves LLM performance on challenging mathematical problems. Furthermore, our experiments demonstrate that structured guidance consistently outperforms unstructured alternatives, and that incorporating asymmetric causal relationships yields greater improvements than using symmetric associations alone.

数学推理因果图大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。