揭示大模型代码推理的内部演化过程，发现答案先‘酿造’后‘解析’。
From Brewing to Resolution: Tracing the Internal Lifecycle of Code Reasoning in LLMs
- 通过层间线性探测与上下文剥离解码，追踪代码推理的内部生命周期。
- 仅41.5%任务最终正确解析，深度调用使成功率从61.1%降至2.5%。
- ‘酿造’阶段稳定存在，但‘解析’成败与模型能力密切相关。
标准准确率无法解释为何大模型能处理变量追踪却在语义等价的循环中失败。本文研究代码推理的内部生命周期：模型先‘酿造’答案，使其在多层前线性可恢复，随后进入四种解析结果之一：已解决、过度处理、错误解决或未解决。理解此生命周期至关重要，因相似准确率可能掩盖根本不同的失败模式，表面评估难以察觉。我们提出双诊断框架，结合层间线性探测与上下文剥离解码（CSD），应用于16个模型（涵盖Qwen、Llama、DeepSeek架构）上的六类代码推理任务。所有任务家族均呈现显著的四种结果分布：总体已解决率仅为41.5%，部分任务低于30%。对结构、深度和操作符的受控实验揭示了特定任务瓶颈：函数调用已解决率随调用深度从1增至3，由61.1%骤降至2.5%。跨架构与规模下，酿造支架保持稳定，归一化酿造时长在24%-42%之间，而解析成功则随能力、规模与训练变化。这表明酿造支架是解码器仅架构下的稳定经验规律，而解析成功率与模型能力正相关。代码开源：https://github.com/euyis1019/llm-brewing
原文摘要 · Abstract (English)
Standard accuracy metrics cannot explain why LLMs handle variable tracking but fail on semantically equivalent loops. We study an internal lifecycle of code reasoning in which models first brew the answer, making it linearly recoverable many layers before it becomes self-decodable, and then diverge into one of four resolution outcomes: Resolved, Overprocessed, Misresolved, or Unresolved. Understanding this lifecycle matters because similar task accuracies can mask fundamentally different failure modes that surface-level evaluation cannot detect. We introduce a dual diagnostic framework pairing layer-wise linear probing with Context-Stripped Decoding (CSD) and apply it to six code-reasoning task families across 16 models spanning Qwen, Llama, and DeepSeek architectures. All four outcomes carry substantial mass in every task family: overall Resolved is only 41.5%, with multiple tasks below 30%. Controlled sweeps over structure, depth, and operators expose task-specific failure bottlenecks: Function Call Resolved plunges from 61.1% to 2.5% as call depth increases from one to three. Across architectures and scales, the brewing scaffold remains stable, with normalized brewing duration 24-42% across all 16 models, while resolution success varies with capability. This indicates that the scaffold is a stable empirical regularity across the tested decoder-only Transformer families, whereas resolution success covaries with capability, scale, and training. Code: https://github.com/euyis1019/llm-brewing
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。