通过保留推理链首尾关键步骤,实现无损加速大模型推理训练。
EPiC: Towards Lossless Speedup for Reasoning Training through Edge-Preserving CoT Condensation
- 只保留推理链的开头和结尾,舍弃中间冗余步骤。
- 训练时间减少超34%,数学题准确率与完整推理持平。
- 适合追求高效训练且不牺牲推理质量的研究者。
大语言模型在链式思维(CoT)监督下展现出强大的推理能力,但来自大型推理模型(如DeepSeek-R1)的长而冗长的CoT轨迹显著增加了训练成本。本文研究资源高效的推理训练中的CoT压缩问题,旨在剪枝中间推理步骤,在缩短推理链长度的同时保持答案准确性和推理连贯性。基于实证分析,我们发现推理过程通常包含三个阶段:问题理解、探索和解题收敛;保留初始问题理解阶段(富含反思线索)和最终解题收敛阶段即可实现无损推理监督。为此,提出边缘保持压缩方法EPiC,仅保留每条CoT的首尾片段,舍弃中间部分,类比于保留推理轨迹的‘边缘’,以维持逻辑连续性。在Qwen和LLaMA多个模型家族及多基准测试上验证,EPiC使训练时间减少超过34%,在MATH500上达到与完整CoT监督相当的无损推理准确率。据我们所知,这是首个探索思想级CoT压缩以实现高效推理模型蒸馏的研究。
原文摘要 · Abstract (English)
Large language models (LLMs) have shown remarkable reasoning capabilities when trained with chain-of-thought (CoT) supervision. However, the long and verbose CoT traces, especially those distilled from large reasoning models (LRMs) such as DeepSeek-R1, significantly increase training costs during the distillation process, where a non-reasoning base model is taught to replicate the reasoning behavior of an LRM. In this work, we study the problem of CoT condensation for resource-efficient reasoning training, aimed at pruning intermediate reasoning steps (i.e., thoughts) in CoT traces, enabling supervised model training on length-reduced CoT data while preserving both answer accuracy and the model's ability to generate coherent reasoning. Our rationale is that CoT traces typically follow a three-stage structure: problem understanding, exploration, and solution convergence. Through empirical analysis, we find that retaining the structure of the reasoning trace, especially the early stage of problem understanding (rich in reflective cues) and the final stage of solution convergence, is sufficient to achieve lossless reasoning supervision. To this end, we propose an Edge-Preserving Condensation method, EPiC, which selectively retains only the initial and final segments of each CoT trace while discarding the middle portion. This design draws an analogy to preserving the "edge" of a reasoning trajectory, capturing both the initial problem framing and the final answer synthesis, to maintain logical continuity. Experiments across multiple model families (Qwen and LLaMA) and benchmarks show that EPiC reduces training time by over 34% while achieving lossless reasoning accuracy on MATH500, comparable to full CoT supervision. To the best of our knowledge, this is the first study to explore thought-level CoT condensation for efficient reasoning model distillation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。