用连续隐状态提升推理效率,保留语言模型核心优势。
Latent Reasoning with Normalizing Flows

- 用归一化流在模型内部建模连续隐式推理过程。
- 代码生成任务上通过率更高,中间计算成本显著降低。
- 适合需要高效推理的场景,如实时生成与强化学习优化。
大语言模型常通过生成显式的思维链(CoT)提升推理能力,但文本形式的思维链迫使计算以离散、串行、通信导向的词元流进行,即使底层更新是语义性、不确定或部分形成的。隐式推理通过紧凑的连续状态执行中间计算,提供更高带宽。然而现有方法常牺牲自回归语言模型中使CoT有效的关键优势:原生左到右生成、概率采样、支持KV缓存解码和可计算的似然估计。本文提出NF-CoT,一种保留这些优势的隐式推理框架,通过在语言模型主干中引入类TARFlow的归一化流,定义从显式CoT中提炼出的紧凑连续隐状态上的可计算概率模型。连续思考位置由归一化流头生成,文本位置由标准语言模型头在同一因果流中生成。该设计实现了隐式思考的精确似然,支持使用原始KV缓存的概率左到右解码,并可在隐式推理空间直接进行策略梯度优化。在代码生成基准测试中,NF-CoT在通过率上优于显式CoT和先前隐式推理基线,同时大幅降低中间推理开销。
原文摘要 · Abstract (English)
Large language models often improve reasoning by generating explicit chain-of-thought (CoT), demonstrating the importance of intermediate computation. However, textual CoT forces this computation through a discrete, serial, and communication-oriented token stream: each reasoning step must be verbalized before the model can proceed, even when the underlying update is semantic, uncertain, or only partially formed. Latent reasoning offers a higher-bandwidth alternative by performing intermediate computation in compact continuous states before committing to text. Yet existing latent-reasoning methods often sacrifice key advantages that make CoT effective in autoregressive language models, including native left-to-right generation, probabilistic sampling, compatibility with KV-cache decoding, and tractable likelihood estimation. We propose NF-CoT, a latent reasoning framework that preserves these advantages by modeling continuous thoughts with normalizing flows. NF-CoT instantiates a TARFlow-style normalizing flow inside the LLM backbone, defining a tractable probability model over compact continuous thoughts distilled from explicit CoT. Continuous-thought positions are generated by an NF head, while text positions are generated by the standard LM head within the same causal stream. This design provides exact likelihoods for latent thoughts, enables probabilistic left-to-right decoding with the original KV cache, and supports direct policy-gradient optimization in the latent reasoning space. On code-generation benchmarks, NF-CoT improves pass rates over explicit-CoT and prior latent-reasoning baselines while substantially reducing intermediate-reasoning cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。