通过压缩推理路径提升大模型推理速度，精度损失仅1.2%。
Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning
- 用近期生成内容评估重要性，定期压缩键值缓存
- 在QwQ-32B上提速1.6倍，AIME 2024准确率下降1.2%
- 无需训练，适合部署高精度逻辑推理模型的场景
近期面向推理的语言模型通过生成较长的中间推理路径来获得高准确率。然而，长推理路径显著增加内存占用并降低生成吞吐量，限制了此类模型的实际部署。我们提出无需训练的推理路径压缩（RPC）方法，利用推理路径中的语义稀疏性加速推理。该方法通过选择器窗口计算最近生成查询的重要性得分，周期性地保留高分缓存条目，从而压缩键值缓存。实验表明，与完整KV缓存相比，RPC使QwQ-32B的生成吞吐量提升最高达1.60倍，在AIME 2024基准测试上准确率仅下降1.2%。结果表明，推理轨迹中的语义稀疏性可被有效利用进行压缩，为高效部署推理型大模型提供了可行路径。代码已公开于https://github.com/jiwonsong-dev/ReasoningPathCompression。
原文摘要 · Abstract (English)
Recent reasoning-focused language models achieve high accuracy by generating lengthy intermediate reasoning paths before producing final answers. While this approach is effective in solving problems that require logical thinking, long reasoning paths significantly increase memory usage and reduce throughput of token generation, limiting the practical deployment of such models. We propose Reasoning Path Compression (RPC), a training-free method that accelerates inference by leveraging the semantic sparsity of reasoning paths. RPC periodically compresses the KV cache by retaining cache entries that receive high importance score, which are computed using a selector window composed of recently generated queries. Experiments show that RPC improves generation throughput of QwQ-32B by up to 1.60$\times$ compared to the inference with full KV cache, with an accuracy drop of 1.2\% on the AIME 2024 benchmark. Our findings demonstrate that semantic sparsity in reasoning traces can be effectively exploited for compression, offering a practical path toward efficient deployment of reasoning LLMs. Our code is available at https://github.com/jiwonsong-dev/ReasoningPathCompression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。