仅用1.7万条示例,模型就能高效学会复杂推理,结构比内容更重要。
LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!

- 用少量示范数据和低秩微调,让大模型掌握长链推理能力。
- 在AIME2024上达到56.7%准确率,比基础版提升40个百分点。
- 推理步骤的逻辑结构远比内容正确性重要,适合优化训练效率。
大型推理模型通过包含反思、回溯和自验证的长链思维(Long CoT)解决复杂问题,但其训练方法与数据需求尚不明确。本研究发现,仅需17,000条长链思维样本,通过数据高效的监督微调(SFT)与参数高效的低秩适配(LoRA),Qwen2.5-32B-Instruct模型即可在数学与编码基准上显著提升性能,包括在AIME 2024上达到56.7%(+40.0%)、LiveCodeBench上达57.0%(+8.1%),接近专有o1-preview模型的44.6%与59.1%表现。更重要的是,我们发现长链思维的结构对学习至关重要,而单个推理步骤的内容影响极小:即使使用错误样本或移除关键词,性能仅下降3.2%;但若打乱或删除推理步骤,准确率将大幅下降。这深化了对如何激发大模型推理能力的理解,并为高效训练下一代推理模型提供关键指导。该成果为此前发布的Sky-T1-32B-Preview模型的学术基础。代码已开源于https://github.com/NovaSky-AI/SkyThought。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) tackle complex reasoning problems by following long chain-of-thoughts (Long CoT) that incorporate reflection, backtracking, and self-validation. However, the training techniques and data requirements to elicit Long CoT remain poorly understood. In this work, we find that a Large Language model (LLM) can effectively learn Long CoT reasoning through data-efficient supervised fine-tuning (SFT) and parameter-efficient low-rank adaptation (LoRA). With just 17k long CoT training samples, the Qwen2.5-32B-Instruct model achieves significant improvements on a wide range of math and coding benchmarks, including 56.7% (+40.0%) on AIME 2024 and 57.0% (+8.1%) on LiveCodeBench, competitive to the proprietary o1-preview model's score of 44.6% and 59.1%. More importantly, we find that the structure of Long CoT is critical to the learning process, whereas the content of individual reasoning steps has minimal impact. Perturbations affecting content, such as training on incorrect samples or removing reasoning keywords, have little impact on performance. In contrast, structural modifications that disrupt logical consistency in the Long CoT, such as shuffling or deleting reasoning steps, significantly degrade accuracy. For example, a model trained on Long CoT samples with incorrect answers still achieves only 3.2% lower accuracy compared to training with fully correct samples. These insights deepen our understanding of how to elicit reasoning capabilities in LLMs and highlight key considerations for efficiently training the next generation of reasoning models. This is the academic paper of our previous released Sky-T1-32B-Preview model. Codes are available at https://github.com/NovaSky-AI/SkyThought.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。