arXiv:2601.07180cs.CL2026-01被引 2

让大模型推理更高效,减少冗余步骤。

Structured Reasoning for Large Language Models

  • 将推理拆解为生成-验证-修订三阶段,结构化提升可控性。
  • 实验显示推理长度减少最高50%,且自验证能力显著增强。
  • 适合追求高效推理、减少资源消耗的AI开发者使用。

大语言模型通过生成长链条思维取得优异表现,但更长的推理轨迹常伴随冗余或无效步骤。例如,模型即使已得出正确答案仍会进行不必要的验证与修正。这一局限源于推理路径的非结构化特性及对关键推理能力缺乏针对性监督。为此,我们提出结构化推理(SCR)框架,将推理轨迹解耦为显式、可评估、可训练的组件。主要采用生成-验证-修订范式,构建结构化训练数据,并引入动态终止监督,指导模型判断何时终止推理。为避免不同推理能力学习信号相互干扰,采用分两阶段的渐进式强化学习策略:第一阶段聚焦初始生成与自验证,第二阶段专注修订。在三种基线模型上的大量实验表明,SCR显著提升了推理效率与自验证能力;相比现有推理范式,输出令牌长度最高减少50%。

原文摘要 · Abstract (English)

Large language models (LLMs) achieve strong performance by generating long chains of thought, but longer traces always introduce redundant or ineffective reasoning steps. One typical behavior is that they often perform unnecessary verification and revisions even if they have reached the correct answers. This limitation stems from the unstructured nature of reasoning trajectories and the lack of targeted supervision for critical reasoning abilities. To address this, we propose Structured Reasoning (SCR), a framework that decouples reasoning trajectories into explicit, evaluable, and trainable components. We mainly implement SCR using a Generate-Verify-Revise paradigm. Specifically, we construct structured training data and apply Dynamic Termination Supervision to guide the model in deciding when to terminate reasoning. To avoid interference between learning signals for different reasoning abilities, we adopt a progressive two-stage reinforcement learning strategy: the first stage targets initial generation and self-verification, and the second stage focuses on revision. Extensive experiments on three backbone models show that SCR substantially improves reasoning efficiency and self-verification. Besides, compared with existing reasoning paradigms, it reduces output token length by up to 50%.

大模型推理结构化推理效率优化强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。