通过图结构剪枝减少大模型推理中的重复思考，提升效率不降精度。
Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMs
- 将线性思维链转为有向无环图，用双策略剪枝冗余反思分支和重复验证。
- 平均减少42%的推理词数，准确率保持不变或提升。
- 适合需要高效推理的场景，如实时问答、资源受限部署。
通过强化学习扩展思维链（CoT）已被广泛用于提升大语言模型的推理能力。然而，由于奖励信号稀疏，容易引发过度思考问题，即生成冗余的中间推理内容。本文认为冗余的主要来源是低效的反思行为，表现为两种模式：泛化反思（对所有步骤进行低效检查）与重复反思（反复验证已确定结论）。为此，提出基于图结构的思维链优化框架：将线性思维链转换为带显式依赖边的有向无环图（DAG），设计双层剪枝策略——分支级剪枝移除贡献弱的反思分支，深度级剪枝消除后期重复验证。通过三阶段训练流程实现行为蒸馏：(1) SFT 在精简轨迹上初始化策略；(2) DPO 偏好正确且更简洁的推理路径；(3) GRPO 结合长度惩罚联合优化答案正确性与效率。实验表明，该方法在保持或提升准确率的同时，平均减少42%的推理词数。
原文摘要 · Abstract (English)
Extending CoT through RL has been widely used to enhance the reasoning capabilities of LLMs. However, due to the sparsity of reward signals, it can also induce undesirable thinking patterns such as overthinking, i.e., generating redundant intermediate reasoning content. In this work, we argue that a major source of such redundancy is inefficient reflection, which often manifests in two problematic patterns: Indiscriminate Reflection, where the model performs broad, low-impact checks throughout reasoning, and Repetitive Reflection, where it repeatedly re-verifies an already established conclusion. To address this, we introduce a graph-based CoT optimization framework. Specifically, we convert each linear CoT into a directed acyclic graph (DAG) with explicit dependency edges, and design a dual pruning strategy: branch-level pruning removes weakly contributing reflection branches, while depth-level pruning eliminates late-stage re-verification. We distill this behavior via a three-stage pipeline: (1) SFT to initialize the policy on pruned concise traces, (2) DPO to prefer correct but less redundant trajectories, and (3) GRPO with length penalty to jointly optimize answer correctness and efficiency. Experiments show that our approach reduces the average reasoning tokens by 42\% while maintaining or improving accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。