arXiv:2605.28742cs.AI2026-05

通过对比成功与失败推理路径,用少量样本快速提升模型推理能力。

CORE: Contrastive Reflection Enables Rapid Improvements in Reasoning

论文配图:CORE: Contrastive Reflection Enables Rapid Improvements in Reasoning
图 1 · 摘自论文原文
  • 不依赖参数更新,通过对比推理过程生成改进策略
  • 仅用5个样本和少量推演即在多数任务上表现最优
  • 输出可解释的自然语言洞察,节省上下文空间

语言模型可利用可验证奖励在多种推理任务中自我改进。然而,现有参数化(如RLVR)和非参数化(如提示优化)方法通常需数百训练样本和数千次模型推演,成本高昂甚至不可行。为此,我们提出非参数学习算法对比反思(CORE),通过比较过往推理轨迹生成洞见:描述推理策略与约束的简短自然语言片段,捕捉成功与失败尝试的差异。在四项推理任务中,CORE展现出比参数化(GRPO)和非参数化(GEPA、情景RAG、MemRL)方法更快的改进速度,且推演次数更少。在固定推演预算下,仅使用5个训练样本时,CORE在多数任务数据场景中表现最强。此外,相比非参数基线,CORE显著更高效,所需提示词更少,同时将知识以紧凑可解释的自然语言洞察形式存储。结果表明,将成功与失败推理轨迹的差异提炼为抽象有用洞见,是比权重更新、提示优化或直接复用存储轨迹更高效、更可解释的模型自提升路径。

原文摘要 · Abstract (English)

Language models can use verifiable rewards to improve at a wide variety of reasoning tasks. However, both parametric (e.g. RLVR) and non-parametric (e.g. prompt optimization) approaches to doing so typically require hundreds of training samples and thousands of model rollouts, making them expensive in the best case and intractable in the worst. To address this challenge, we introduce Contrastive Reflection (CORE), a non-parametric learning algorithm that compares past reasoning traces to generate insights: short natural-language descriptions of reasoning strategies and constraints that capture differences between successful and unsuccessful problem attempts. Across four reasoning tasks, we demonstrate that CORE enables more rapid improvement than both parametric (GRPO) and non-parametric (GEPA, episodic RAG, and MemRL) methods, while using fewer rollouts. Under fixed rollout budgets with as few as five training samples, CORE achieves the strongest performance in most task-data regimes. Finally, we highlight how CORE is substantially more context-efficient than non-parametric baselines, requiring fewer prompt tokens while storing learned knowledge as compact, interpretable natural-language insights. Our results therefore suggest that distilling contrasts between successful and unsuccessful reasoning traces into abstract and useful insights can provide a more efficient and interpretable route to model self-improvement than weight updates, prompt optimization, or direct reuse of stored reasoning traces.

推理增强自我改进非参数方法可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。