按解题逻辑排序示范例，提升大模型复杂推理能力
Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoning
- 基于解题逻辑分析筛选示范例，而非简单相关性
- 按解题步骤数从易到难排序，符合循序渐进原则
- 在多个基准上超越传统ICL，尤其适合复杂推理任务
上下文学习（ICL）可显著增强大语言模型（LLMs）的复杂推理能力，关键在于示范例的选择与排序。现有方法多依赖简单特征衡量示例相关性，但这些特征难以反映示例间的内在联系。本文提出一种由解题逻辑引导的课程式ICL策略：基于BREAK数据集构建解题逻辑指令集，并微调语言模型以分析示例的解题逻辑；根据解题步骤数评估难度，选择合适示范例并按从易到难顺序排列，作为上下文提示。在多个基准上的实验结果表明，该方法在性能与效率上均优于现有ICL方法，有效提升了LLMs的复杂推理能力。
原文摘要 · Abstract (English)
In-context learning (ICL) can significantly enhance the complex reasoning capabilities of large language models (LLMs), with the key lying in the selection and ordering of demonstration examples. Previous methods typically relied on simple features to measure the relevance between examples. We argue that these features are not sufficient to reflect the intrinsic connections between examples. In this study, we propose a curriculum ICL strategy guided by problem-solving logic. We select demonstration examples by analyzing the problem-solving logic and order them based on curriculum learning. Specifically, we constructed a problem-solving logic instruction set based on the BREAK dataset and fine-tuned a language model to analyze the problem-solving logic of examples. Subsequently, we selected appropriate demonstration examples based on problem-solving logic and assessed their difficulty according to the number of problem-solving steps. In accordance with the principles of curriculum learning, we ordered the examples from easy to hard to serve as contextual prompts. Experimental results on multiple benchmarks indicate that our method outperforms previous ICL approaches in terms of performance and efficiency, effectively enhancing the complex reasoning capabilities of LLMs. Our project will be released at https://github.com/maxuetao/CurriculumICL
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。