将自回归推理蒸馏为无声思维，提升大模型推理效率
DART: Distilling Autoregressive Reasoning to Silent Thought
- 通过自蒸馏框架，让模型从逐步推理转为直接生成答案
- 在不增加推理延迟的前提下，性能超越现有非自回归方法
- 适合对响应速度要求高的场景，如实时对话系统
链式思维（CoT）推理显著提升了大语言模型解决复杂任务的能力。然而，其自回归机制带来显著计算开销，限制了在低延迟场景中的部署。为此，我们提出DART（Distilling Autoregressive Reasoning to Silent Thought），一种自蒸馏框架，使大模型能够用非自回归的无声思维（ST）替代自回归CoT。DART引入两条训练路径：一条用于传统推理（CoT路径），另一条用于直接从少量ST token生成答案（ST路径）。ST路径采用轻量级推理演进模块（REM），使隐藏状态与CoT路径对齐，推动ST token演化为信息丰富的嵌入表示。推理阶段仅激活ST路径，通过演进后的ST token直接输出答案。大量实验表明，DART在不增加额外推理延迟的情况下，显著优于现有非自回归基线，是高效推理的可行替代方案。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) reasoning has significantly advanced Large Language Models (LLMs) in solving complex tasks. However, its autoregressive paradigm leads to significant computational overhead, hindering its deployment in latency-sensitive applications. To address this, we propose \textbf{DART} (\textbf{D}istilling \textbf{A}utoregressive \textbf{R}easoning to Silent \textbf{T}hought), a self-distillation framework that enables LLMs to replace autoregressive CoT with non-autoregressive Silent Thought (ST). Specifically, DART introduces two training pathways: the CoT pathway for traditional reasoning and the ST pathway for generating answers directly from a few ST tokens. The ST pathway utilizes a lightweight Reasoning Evolvement Module (REM) to align its hidden states with the CoT pathway, enabling the ST tokens to evolve into informative embeddings. During inference, only the ST pathway is activated, leveraging evolving ST tokens to deliver the answer directly. Extensive experimental results demonstrate that DART offers significant performance gains compared with existing non-autoregressive baselines without extra inference latency, serving as a feasible alternative for efficient reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。