arXiv:2608.02585cs.LGcs.CL2026-08

让大模型在推理时直接优化内部状态,提升准确率并看清每一步逻辑来源。

GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning

论文配图:GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning
图 1 · 摘自论文原文
  • 在Transformer中间层插入可优化的隐状态,通过反向传播直接分配奖励梯度。
  • 平均准确率达64.5%,比思维链提示高6.6个百分点,优于现有最强方法2.4点。
  • 能定位关键推理节点,适合需要可解释性与稳定性的实际部署场景。

基于优化的隐状态推理通过在测试时优化特定实例的连续状态来提升大语言模型输出,同时保持模型参数不变。然而,现有方法通常通过解码的词元连接状态与推理轨迹,导致序列级信用分配间接,难以揭示隐状态更新如何影响后续推理。我们提出GradCuit(梯度通过电路),在提示与生成延续之间的选定Transformer层插入可优化的隐状态。因果自注意力为每个延续词元的对数概率提供一条可微路径,使其能直接接收来自整个延续的奖励加权梯度,从而将反馈精准传递至隐状态。在五个指令微调骨干模型、三个推理基准和两种答案格式下,GradCuit平均准确率达64.5%,比思维链提示高出6.6个百分点,优于最强对比方法2.4个百分点。该方法还表现出更强鲁棒性:在七种学习率设置下,始终优于LatentSeek,准确率标准差从1.53降至0.82;其随机游走变体仍具备竞争力。可解释性方面,词元级梯度归因显示隐状态影响集中于推理连接词元,层分析表明早期到中期的Transformer层是最佳优化空间。通过直接从结果反馈优化内部推理过程,GradCuit开辟了鲁棒且可解释的测试时扩展新路径,使大模型不仅能重生成、采样或重排序输出,更能动态调整推理方式。

原文摘要 · Abstract (English)

Optimization-based latent reasoning improves large language model outputs by optimizing instance-specific continuous states at test time while keeping model parameters frozen. Existing methods, however, typically connect these states to the reasoning trajectory through decoded tokens, making sequence-level credit assignment indirect and obscuring how latent updates shape subsequent reasoning. We introduce GradCuit (gradient through circuit), which inserts optimizable latent states at a selected Transformer layer between the hidden representations of the prompt and the generated continuation. Causal self-attention provides every continuation-token log-probability with a differentiable path to every preceding latent state through the remaining Transformer blocks, enabling reward-weighted gradients from the entire continuation to be assigned directly to the latents. Across five instruction-tuned backbones, three reasoning benchmarks, and two answer formats, GradCuit achieves an average accuracy of 64.5%, outperforming chain-of-thought prompting by 6.6 percentage points and the strongest competing method by 2.4 points. GradCuit also demonstrates greater robustness: across seven learning-rate settings, it consistently outperforms LatentSeek while reducing the standard deviation of accuracy from 1.53 to 0.82, and even its random-walk variant remains competitive with LatentSeek. For interpretability, token-level gradient attribution reveals that latent influence concentrates on reasoning-connector tokens, while layer analysis identifies early-to-middle Transformer layers as the most effective optimization space. By directly optimizing internal reasoning from outcome feedback, GradCuit opens a new axis of robust and interpretable test-time scaling, where LLMs adapt how they reason rather than merely regenerate, sample, or rerank outputs.

测试时推理可解释性梯度优化大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。