通过强化学习压缩推理链条，减少80%以上冗余计算。
TRiMS: Real-Time Tracking of Minimal Sufficient Length for Efficient Reasoning via RL
- 定义最小充分长度MSL，量化推理最短有效长度
- 实测推理 token 减少超80%，准确率小幅提升
- 适合追求高效推理的模型部署与边缘计算场景
大型语言模型通过长链式思维（CoT）序列在复杂推理任务中取得突破，但常导致严重的推理膨胀，造成大量计算冗余。为实现每令牌智能最大化，本文提出理论度量 MSL（Minimal Sufficient Length），严格刻画保持答案正确的最短推理长度。基于独立采样序列给出递归定义，并证明其极限存在性，首次建立推理链压缩的可测量下界。结合主流 CoT 压缩策略分析，识别出逼近 MSL 的关键结构因素。基于此，提出 TRiMS 方法，在训练中采用 GRPO 算法并结合 MSL 估计，通过动态批量聚合和批次标准差计算优势值，缓解训练不稳定性。TRiMS 在所有基准测试中实现超过80%的 CoT token 减少，且准确率略有提升。
原文摘要 · Abstract (English)
Large language models achieve breakthroughs in complex reasoning via long chain-of-thought sequences. However, this often leads to severe reasoning inflation, causing substantial computational redundancy. To maximize Intelligence per Token, we introduce a theoretical metric, MSL-Minimal Sufficient Length. MSL rigorously characterizes the shortest reasoning length that preserves answer correctness. We provide a recursive definition based on independently sampled sequences and prove the existence of its limit, establishing the first measurable lower bound for reasoning-chain compression. Building on an analysis of mainstream CoT compression strategies, we identify key structural factors enabling a model to approach MSL. Based on these insights, we propose TRiMS which employs the GRPO algorithm in conjunction with MSL-based estimation during training, while mitigating instabilities during the training process through dynamic batch aggregation and advantage computation using batch-level standard deviation. TRiMS achieves over 80% CoT token reduction with a minor accuracy boost across all benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。