arXiv:2509.23196cs.CL2025-09被引 1

让推理模型从错误示范中提取有用洞察,提升解题准确率。

From Harm to Help: Turning Reasoning In-Context Demos into Assets for Reasoning LMs

  • 将示例转化为可复用的推理洞察,分步引导模型解题
  • 在AIME'25上使GPT-4.1准确率提升14.0%,o1-mini提升2.7%
  • 适用于各类开源与闭源大模型,尤其改善少样本推理

近期基于验证器强化学习训练的推理大模型(RLMs)在使用少量示例思维链(CoT)时表现反而不如直接作答。本文利用DeepSeek-R1生成的高质量推理轨迹作为示例,发现增加示例数量会持续降低准确率,即使示例本身已最优。分析揭示两大机制:(i) 语义误导,高文本相似性导致模型将目标误认为示例并照搬中间步骤;(ii) 策略迁移失败,模型难以提取有效推理策略并应用于目标问题。为此提出I2S(Insight-to-Solve)方法,通过测试时顺序处理,将示例转化为显式可复用的洞察,并生成针对目标的推理链;可选地进一步自精炼以提升连贯性与正确性(I2S+)。在多个基准测试上的实验表明,I2S与I2S+在开放与闭源模型上均显著优于直接作答和测试时扩展基线。即使对GPT系列也有效:在AIME'25上GPT-4.1提升+14.0%,o1-mini在AIME上提升+2.7%、在GPQA上提升+1.7%,证明通过洞察-精炼-求解框架可高效利用上下文示例。

原文摘要 · Abstract (English)

Recent reasoning LLMs (RLMs), especially those trained with verifier-based reinforcement learning, often perform worse with few-shot CoT than with direct answering. We revisit this paradox using high-quality reasoning traces from DeepSeek-R1 as demonstrations and find that adding more exemplars consistently degrades accuracy, even when demonstrations are optimal. A detailed analysis reveals two mechanisms behind this decline: (i) semantic misguidance, where high textual similarity leads the model to treat the target as the same as the exemplar and to copy intermediate steps verbatim; and (ii) strategy transfer failure, where the model struggles to extract useful reasoning strategies and apply them to target questions. Guided by these, we introduce Insight-to-Solve (I2S), a sequential test-time procedure that turns demonstrations into explicit, reusable insights and derives a target-specific reasoning trace; optionally, the reasoning is self-refined for coherence and correctness (I2S+). Extensive experiments on diverse benchmarks show that I2S and I2S+ consistently outperform both direct answering and test-time scaling baselines across open- and closed-source models. Even for GPT models, our method helps: on AIME'25, GPT-4.1 rises by +14.0%, and o1-mini improves by +2.7% on AIME and +1.7% on GPQA, indicating that in-context demonstrations can be harnessed effectively via insight-refine-solve framework.

推理增强提示工程模型优化少样本学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。