用符号推理验证每步逻辑,生成更可信的思维链数据
ORACLE: Optimizing Reasoning Abilities of Large Language Models via Constraint-Led Synthetic Data Elicitation
- 用大模型生成推理步骤,符号引擎逐步验证
- 在6个推理基准上优于现有方法,提升模型逻辑能力
- 适合需要严谨逻辑训练的AI系统开发者
利用合成推理数据训练大语言模型已成为提升其推理能力的主流方法,但关键在于生成数据的质量。现有方法多基于最终答案正确性筛选合成推理路径,常忽略中间步骤的错误。以往依赖代码执行或符号推理引擎验证,但前者仅适用于编程或数学任务,后者需完整结构化上下文,难以应对自然语言中模糊或不完整的场景。为此,我们提出ORACLE,一种受三段论启发的结构化数据生成框架。该框架结合大模型生成能力与符号监督:由大模型生成分步推理内容,符号推理引擎逐步验证有效性。通过统一提示模板激发模块化推理链,实现细粒度的步骤级验证,从而构建高质量多步推理数据。在六个逻辑、事实与常识推理基准上,ORACLE对多个模型均持续超越强基线。
原文摘要 · Abstract (English)
Training large language models (LLMs) with synthetic reasoning data has become a popular approach to enhancing their reasoning capabilities, while a key factor influencing the effectiveness of this paradigm is the quality of the generated multi-step reasoning data. To generate high-quality reasoning data, many recent methods generate synthetic reasoning paths and filter them based on final answer correctness, often overlooking flaws in intermediate reasoning steps. To enhance the verification of intermediate reasoning steps, prior work primarily resorts to code execution or symbolic reasoning engines. However, code-based validation is restricted to code or mathematical tasks, and reasoning engines require a well-structured and complete context. As a result, existing methods fail to function effectively in natural language reasoning tasks that involve ambiguous or incomplete contexts. In these tasks, synthetic data still lack reliable checks for verifying each reasoning step. To address this challenge, we introduce ORACLE, a structured data generation framework inspired by syllogistic reasoning. ORACLE integrates the generative strengths of LLMs with symbolic supervision: the LLM produces step-wise reasoning contexts, while a symbolic reasoning engine verifies the validity of each intermediate step. By employing a unified prompting template to elicit modular reasoning chains, ORACLE enables fine-grained, step-level validation, facilitating the construction of high-quality multi-step reasoning data. Across six logical, factual, and commonsense reasoning benchmarks, our ORACLE consistently outperforms strong baselines on multiple models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。