arXiv:2601.23027cs.LG2026-01被引 1

让大模型并行思考,提速35%以上,同时保持高准确率。

Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoning

  • 设计并行推理框架,将长链思考拆解为可并行执行的子任务。
  • 在AIME 2024和HMMT 2025上实现与原模型相当的准确率,最长路径降低35%-40%。
  • 适合需要低延迟推理的数学推理场景,如实时答题系统。

长链思维(Long CoT)是当前先进大模型在数学推理等任务中的核心能力,但其生成过程高度串行,导致高延迟。本文提出分而治之思维(DC-CoT),训练模型在推理时充当调度者,识别可并行执行的子任务,并分派执行。目标是在保持高准确率的同时,显著降低响应所需最长路径长度(理论延迟指标)。基于DeepScaleR-1.5B-Preview模型,先用小规模精选示范数据进行监督微调(SFT)以初始化并行能力,但SFT会显著损害准确率。为此,设计多阶段强化学习算法,结合多种数据过滤策略,在恢复准确率的同时减少最长路径长度。在AIME 2024和HMMT 2025等多个基准测试中,DC-CoT达到与原模型相近的准确率,最长路径长度降低35%-40%。代码、SFT数据集及模型已公开于https://github.com/amahankali10/DC_CoT_RL_for_Low_Latency_CoT_with_Parallel_Reasoning。

原文摘要 · Abstract (English)

Long chain-of-thought reasoning (Long CoT) is now fundamental to state-of-the-art LLMs, especially in mathematical reasoning. However, LLM generation is highly sequential, and long CoTs lead to a high latency. We propose to train Divide-and-Conquer CoT (DC-CoT) to reduce the latency. With DC-CoT, the model can act as a director that identifies distinct subtasks that can be performed in parallel in its reasoning process, and then spawns workers to execute the subtasks. Our goal is to achieve high accuracy, with a low longest path length, which is a theoretical measure of the latency needed for the response. We start with a long CoT base model (DeepScaleR-1.5B-Preview), and first use SFT with a small curated demonstration set to initialize its ability to spawn workers in a certain format. Because SFT degrades the accuracy significantly, we design a multi-stage RL algorithm, with various data filtering strategies, to recover the accuracy while decreasing the longest path length. Across several benchmarks including AIME 2024 and HMMT 2025, DC-CoT achieves similar accuracy as DeepScaleR-1.5B-Preview while decreasing longest path length by 35-40%. Our code, SFT dataset and models are publicly available at https://github.com/amahankali10/DC_CoT_RL_for_Low_Latency_CoT_with_Parallel_Reasoning.

推理加速并行计算强化学习数学推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。