用强化学习压缩大模型推理过程，不降性能还省算力。
Long Chain-of-Thought Compression via Fine-Grained Group Policy Optimization
- 分组优化+长度熵权重，精细控制推理链压缩
- 在MATH500等数据集上压缩率超30%且准确率不变
- 适合追求推理效率的AI系统开发者
大型语言模型（LLMs）生成的思维链（CoT）常过于冗长，增加计算开销但性能提升有限。本文提出细粒度分组策略优化（FGO），一种基于强化学习的算法，通过拆分分组并根据长度与熵分配权重，实现高效的CoT压缩。作为分组相对策略优化（GRPO）的改进版本，FGO解决了其两大缺陷：数据利用效率低和熵崩溃问题。我们在多个推理类LLM和基准测试上评估，包括MATH500、AIME24、AMC23和Minerva。实验结果表明，FGO在不降低性能的前提下实现了高效的CoT压缩，并有效克服了GRPO的核心局限。代码已开源：https://github.com/Mr-XcHan/FGO。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often generate unnecessarily verbose Chain-of-Thought (CoT) reasoning that increases computational costs and latency without proportional performance gains. In this paper, we propose Fine-grained Group policy Optimization (FGO), a Reinforcement Learning (RL) algorithm that refines group responses by subdividing them and assigning appropriate weights based on length and entropy, thereby enabling effective CoT compression. Meanwhile, as an enhanced variant of Group Relative Policy Optimization (GRPO), FGO successfully addresses two major limitations of the GRPO: inefficient data utilization and entropy collapse. We evaluate FGO on multiple reasoning LLMs and benchmarks, including MATH500, AIME24, AMC23, and Minerva. Experimental results show that FGO achieves efficient CoT compression without degrading performance, and simultaneously resolves the key limitations of GRPO. Code: https://github.com/Mr-XcHan/FGO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。