arXiv:2601.03682cs.CLcs.AI2026-01ACL

让大模型先学会找逻辑关系,解数学题更准更快

From Implicit to Explicit: Token-Efficient Logical Supervision for Mathematical Reasoning in LLMs

  • 聚焦解题第一步:识别变量与运算,显式训练逻辑理解
  • 在多个数据集上提升准确率,分布内平均高3.2%,分布外高4.6%
  • 训练速度提升4-6倍,用Token减少超80%,适合资源有限场景

近期研究发现,大语言模型在数学问题求解中逻辑推理能力有限,常依赖模式匹配和记忆。我们系统分析此局限,聚焦逻辑关系理解这一核心能力,发现相关错误占错误预测的90%以上,且链式思维监督微调(CoT-SFT)无法有效缓解。为此,我们提出首步逻辑推理(FSLR),一种轻量级训练框架,专注于逻辑关系理解。关键洞察在于:解题的第一步——确定使用哪些变量、执行何种运算——能促使模型直接从题干推导逻辑关系。通过仅训练该孤立步骤,FSLR为逻辑关系理解提供显式监督,而CoT-SFT则将其隐式嵌入完整解题轨迹中。多模型、多数据集实验证明,FSLR在分布内与分布外设置下均显著优于CoT-SFT,平均提升3.2%与4.6%;同时训练速度提升4-6倍,训练Token消耗减少超80%。

原文摘要 · Abstract (English)

Recent studies reveal that large language models (LLMs) exhibit limited logical reasoning abilities in mathematical problem-solving, instead often relying on pattern-matching and memorization. We systematically analyze this limitation, focusing on logical relationship understanding, which is a core capability underlying genuine logical reasoning, and reveal that errors related to this capability account for over 90\% of incorrect predictions, with Chain-of-Thought Supervised Fine-Tuning (CoT-SFT) failing to substantially reduce these errors. To address this bottleneck, we propose First-Step Logical Reasoning (FSLR), a lightweight training framework targeting logical relationship understanding. Our key insight is that the first planning step-identifying which variables to use and which operation to apply-encourages the model to derive logical relationships directly from the problem statement. By training models on this isolated step, FSLR provides explicit supervision for logical relationship understanding, unlike CoT-SFT which implicitly embeds such relationships within complete solution trajectories. Extensive experiments across multiple models and datasets demonstrate that FSLR consistently outperforms CoT-SFT under both in-distribution and out-of-distribution settings, with average improvements of 3.2\% and 4.6\%, respectively. Moreover, FSLR achieves 4-6x faster training and reduces training token consumption by over 80\%.

逻辑推理数学问答高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。