arXiv:2510.08483cs.CLcs.AI2025-10ACL被引 10

解决大模型并行推理中80%冗余问题,实现高效高精度推理。

DeepPrune: Parallel Scaling without Inter-trace Redundancy

  • 用判别模型预测部分推理路径是否等价,动态剪枝重复路径。
  • 相比传统方法节省65.73%~88.50%计算量,准确率仅降3个百分点。
  • 适合追求高效推理的科研与工程应用,尤其在资源受限场景。

并行推理通过同时生成多个思维链(CoT)轨迹来增强大语言模型的推理能力,但其存在严重的计算低效问题——分析显示超过80%的并行推理轨迹最终答案相同,造成大量计算浪费。为此,我们提出DeepPrune框架,通过动态剪枝实现高效并行推理。该方法采用基于分布外数据(AIME 2022、AIME 2023、MATH 500)与过采样技术训练的专用判别模型,可从部分推理轨迹中准确预测答案等价性,在未见推理模型上达到0.7072 AUROC。结合在线贪心聚类算法,动态剪除冗余路径,同时保留答案多样性。在AIME 2024、AIME 2025和GPQA三个挑战性基准上,对多种推理模型的综合评估表明,DeepPrune相较传统共识采样实现65.73%–88.50%的令牌减少,同时保持性能在3个百分点内波动。本工作为高效并行推理建立了新标准,使高性能推理更经济可行。代码与数据详见:https://deepprune.github.io/。

原文摘要 · Abstract (English)

Parallel scaling has emerged as a powerful paradigm to enhance reasoning capabilities in large language models (LLMs) by generating multiple Chain-of-Thought (CoT) traces simultaneously. However, this approach introduces significant computational inefficiency due to inter-trace redundancy -- our analysis reveals that over 80% of parallel reasoning traces yield identical final answers, representing substantial wasted computation. To address this critical efficiency bottleneck, we propose DeepPrune, a novel framework that enables efficient parallel scaling through dynamic pruning. Our method features a specialized judge model trained with out-of-distribution data (AIME 2022, AIME 2023, and MATH 500) using oversampling techniques to accurately predict answer equivalence from partial reasoning traces, achieving 0.7072 AUROC on unseen reasoning models. Combined with an online greedy clustering algorithm that dynamically prunes redundant paths while preserving answer diversity. Comprehensive evaluations across three challenging benchmarks (AIME 2024, AIME 2025, and GPQA) and multiple reasoning models demonstrate that DeepPrune achieves remarkable token reduction of 65.73%--88.50% compared to conventional consensus sampling, while maintaining competitive accuracy within 3 percentage points. Our work establishes a new standard for efficient parallel reasoning, making high-performance reasoning more efficient. Our code and data are here: https://deepprune.github.io/.

并行推理高效计算大模型剪枝

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。