arXiv:2505.19641cs.AIcs.CL2025-05NeurIPS被引 44

构建可验证的逻辑推理数据集,提升大模型泛化推理能力

SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyond

  • 通过可控合成生成35类逻辑任务数据,支持难度与数量调节
  • 在7B/32B模型上训练后,逻辑推理性能超越开源基线6分
  • 混合数学编码数据能显著提升训练效率和泛化表现

近期如OpenAI-o1和DeepSeek R1等进展表明强化学习(RL)可有效增强大语言模型(LLM)的推理能力。尽管开源复现多集中于数学与编程领域,通用推理能力的方法与资源仍显不足,主要受限于难以获取多样且可验证的推理数据。本文提出逻辑推理是通用推理的基础,为此构建了SynLogic数据合成框架与数据集,可规模化生成涵盖35类逻辑推理任务的数据。所有样本可通过简单规则验证,适合用于具有可验证奖励的强化学习训练。实验验证了基于7B与32B模型在SynLogic上的训练效果:其逻辑推理性能在BBEH上超越DeepSeek-R1-Distill-Qwen-32B达6分;将该数据与数学、编程任务混合训练,不仅提升各领域训练效率,还显著增强推理泛化能力,混合模型在多个基准上优于DeepSeek-R1-Zero-Qwen-32B。相关代码与数据已开源。

原文摘要 · Abstract (English)

Recent advances such as OpenAI-o1 and DeepSeek R1 have demonstrated the potential of Reinforcement Learning (RL) to enhance reasoning abilities in Large Language Models (LLMs). While open-source replication efforts have primarily focused on mathematical and coding domains, methods and resources for developing general reasoning capabilities remain underexplored. This gap is partly due to the challenge of collecting diverse and verifiable reasoning data suitable for RL. We hypothesize that logical reasoning is critical for developing general reasoning capabilities, as logic forms a fundamental building block of reasoning. In this work, we present SynLogic, a data synthesis framework and dataset that generates diverse logical reasoning data at scale, encompassing 35 diverse logical reasoning tasks. The SynLogic approach enables controlled synthesis of data with adjustable difficulty and quantity. Importantly, all examples can be verified by simple rules, making them ideally suited for RL with verifiable rewards. In our experiments, we validate the effectiveness of RL training on the SynLogic dataset based on 7B and 32B models. SynLogic leads to state-of-the-art logical reasoning performance among open-source datasets, surpassing DeepSeek-R1-Distill-Qwen-32B by 6 points on BBEH. Furthermore, mixing SynLogic data with mathematical and coding tasks improves the training efficiency of these domains and significantly enhances reasoning generalization. Notably, our mixed training model outperforms DeepSeek-R1-Zero-Qwen-32B across multiple benchmarks. These findings position SynLogic as a valuable resource for advancing the broader reasoning capabilities of LLMs. We open-source both the data synthesis pipeline and the SynLogic dataset at https://github.com/MiniMax-AI/SynLogic.

逻辑推理强化学习数据合成大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。