arXiv:2510.14620cs.CLcs.AI2025-10被引 1

用代码生成数列数据,让大模型学会自己试错和纠错,提升归纳推理能力。

Code-driven Number Sequence Calculation: Enhancing the inductive Reasoning Abilities of Large Language Models

  • 将数列问题转化为算法求通项任务,自动生成带反思的训练数据
  • 在多个推理任务上提升性能,且保持对未知数据的泛化能力
  • 适合想提升模型自主推理能力的研究者和开发者

大语言模型在推理任务中表现突出,其中归纳推理因更贴近人类学习方式而受到关注。然而现有研究面临两大挑战:一是数据多聚焦表层规律,缺乏深层模式;二是仅通过简单提示或微调,未提供精确思维过程或难度控制。为此,我们提出 extit{CodeSeq},一个基于数列的合成后训练数据集。将数列封装为算法问题以发现通项,定义通项生成(GTG)任务。通过反思失败测试案例并迭代修正,构建监督微调数据,使模型学会自主生成案例与自我检验。同时引入基于可解性(由通过率估计)与自生成成功率双重评估的强化学习奖励机制,让模型从成功与失败中更高效学习。实验表明,使用 extit{CodeSeq} 训练的模型在多种推理任务中表现提升,并维持了对分布外数据的鲁棒性。

原文摘要 · Abstract (English)

Large language models (LLMs) make remarkable progress in reasoning tasks. Among different reasoning modes, inductive reasoning, due to its better alignment with human learning, attracts increasing interest. However, research on inductive reasoning faces certain challenges. First, existing inductive data mostly focuses on superficial regularities while lacking more complex internal patterns. Second, current works merely prompt LLMs or finetune on simple prompt-response pairs, but do not provide precise thinking processes nor implement difficulty control. Unlike previous work, we address these challenges by introducing \textit{CodeSeq}, a synthetic post-training dataset built from number sequences. We package number sequences into algorithmic problems to discover their general terms, defining a general term generation (GTG) task correspondingly. Our pipeline generates supervised finetuning data by reflecting on failed test cases and incorporating iterative corrections, thereby teaching LLMs to learn autonomous case generation and self-checking. Additionally, it leverages reinforcement learning with a novel Case-Synergy Solvability Scaling Reward based on both solvability, estimated from the problem pass rate, and the success rate of self-directed case generation, enabling models to learn more effectively from both successes and failures. Experimental results show that the models trained with \textit{CodeSeq} improve on various reasoning tasks and can preserve the models' OOD performance.

归纳推理代码生成强化学习大模型训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。