让小模型学会生成高质量解题计划，效果超过大模型的零样本推理。
CRISP: Complex Reasoning with Interpretable Step-based Plans
- 构建可解释的分步计划数据集，用大模型自动生成并验证其有效性。
- 微调小模型在该数据集上，生成计划质量超越大模型的少样本提示。
- 跨领域实验显示，学过一个领域的计划能力可迁移到另一领域。
大型语言模型（LLMs）在解决复杂问题时需要更强的推理能力。尽管思维链（Chain-of-Thought, CoT）有所进展，但在许多领域仍显不足。一种有前景的替代方案是生成显式的高层次计划，但现有方法大多假设模型仅通过少样本提示即可生成有效计划，无需额外训练。本文挑战这一假设，提出CRISP（Complex Reasoning with Interpretable Step-based Plans），一个涵盖数学推理与代码生成的多领域高层计划数据集。这些计划由大模型自动生成，并通过大模型评分（内在验证）和下游任务表现（外在验证）双重检验。实验证明，对小型模型在CRISP上微调后，其生成的计划质量优于使用少样本提示的大模型，且显著超越传统思维链。此外，跨领域评估显示，在某一领域微调可提升另一领域的计划生成能力，表明所学规划能力具备良好泛化性。
原文摘要 · Abstract (English)
Recent advancements in large language models (LLMs) underscore the need for stronger reasoning capabilities to solve complex problems effectively. While Chain-of-Thought (CoT) reasoning has been a step forward, it remains insufficient for many domains. A promising alternative is explicit high-level plan generation, but existing approaches largely assume that LLMs can produce effective plans through few-shot prompting alone, without additional training. In this work, we challenge this assumption and introduce CRISP (Complex Reasoning with Interpretable Step-based Plans), a multi-domain dataset of high-level plans for mathematical reasoning and code generation. The plans in CRISP are automatically generated and rigorously validated--both intrinsically, using an LLM as a judge, and extrinsically, by evaluating their impact on downstream task performance. We demonstrate that fine-tuning a small model on CRISP enables it to generate higher-quality plans than much larger models using few-shot prompting, while significantly outperforming Chain-of-Thought reasoning. Furthermore, our out-of-domain evaluation reveals that fine-tuning on one domain improves plan generation in the other, highlighting the generalizability of learned planning capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。