arXiv:2605.28008cs.AIcs.LG2026-05被引 1

研究压缩思维链在大模型微调中的效果,发现不同压缩方式影响数据需求和泛化能力。

Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Training

论文配图:Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Training
图 1 · 摘自论文原文
  • 提出显式、组合、隐式三种思维链压缩类型,系统分析其差异
  • 粗粒度压缩需更多数据,组合型更受益于数据量提升
  • 强化学习能分解压缩步骤,适合长序列任务的通用性提升

大语言模型可通过长思维链解决复杂问题,但性能与令牌开销之间存在权衡。为缓解此问题,监督微调常使用压缩思维链数据,将完整推理过程简化为紧凑形式。然而,压缩数据对后训练的影响尚不明确。本文提出思维链分类体系:显式(输出所有操作)、组合(多步合并为一步)、隐式(省略中间步骤)。通过构建可控合成组合推理任务,系统测试了不同模型规模与家族下的表现。结果表明:(i) 粗粒度压缩需更多微调数据;(ii) 相较于显式,组合与隐式压缩更依赖数据扩展,组合型受益于数据重复,隐式易导致记忆;(iii) 与微调不同,基于可验证奖励的强化学习(RLVR)可分解微调中学习到的压缩步骤;(iv) 单向思维链顺序在长序列任务上具有更强泛化能力。研究为资源受限场景下的思维链设计提供指导,并揭示了微调与强化学习的作用机制。

原文摘要 · Abstract (English)

Large language models (LLMs) can now solve complex problems through long chain-of-thought (CoT) reasoning, but the trade-off between performance and token cost remains a central challenge. To address this issue, supervised fine-tuning (SFT) often uses compressed reasoning data, where CoT traces are shortened into compact forms. However, the effect of such compressed reasoning data on post-training remains poorly understood. In this paper, we propose a taxonomy of CoT consisting of Explicit CoT, which outputs all operations without aggregation, Composed CoT, which combines multiple operations into a single step, and Implicit CoT, which omits intermediate operations. As a controlled mechanistic study, we construct a synthetic compositional reasoning task that allows controlled variation of difficulty, compression granularity, and data size, and conducted a comprehensive set of experiments across different model families and sizes. Notably, we find that (i) coarser CoT requires more SFT data, (ii) compared with Explicit CoT, Composed CoT and Implicit CoT benefit more from data scaling, while Composed CoT benefits from data repetition and Implicit CoT tends to lead to memorization, (iii) unlike SFT, subsequent reinforcement learning (RL) with verifiable rewards (RLVR) decomposes compressed steps learned during SFT, and (iv) unidirectional CoT ordering shows stronger generalization on longer sequential tasks. Our findings provide implications for CoT design under data resource constraints and offer important insights into the mechanisms of SFT and RL in LLM post-training.

思维链模型微调强化学习数据效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。