量化大模型推理过程的不可见深度，揭示其思维外显程度。
Quantifying the Necessity of Chain of Thought through Opaque Serial Depth
- 定义'不透明序列深度'，衡量模型内部推理的不可见长度。
- 计算出Gemma 3模型不透明深度上限，发现其推理难以完全外显。
- 开源工具可评估任意神经网络的深度，适用于专家混合模型分析。
大型语言模型倾向于将其推理过程外显为思维链，这使得思维链成为监控的理想目标。这一现象部分源于Transformer架构的固有特性：足够长的序列认知必须通过思维链传递（Korbak等，2025）。本文通过‘不透明序列深度’的概念进行形式化，该指标指在不使用可解释中间步骤（如思维链）的前提下，模型能够完成的最长计算长度。基于此，我们计算了Gemma 3模型的不透明序列深度上界，并给出了其他架构的渐近结果。同时，我们开源了一种自动化方法，可对任意神经网络计算不透明序列深度上界，并用该方法证明混合专家模型的深度普遍低于密集模型。总体而言，这些结果表明不透明序列深度是理解模型是否具备显著且未外显推理能力的重要工具。
原文摘要 · Abstract (English)
Large language models (LLMs) tend to externalize their reasoning in their chain of thought, making the chain of thought a good target for monitoring. This is partially an inherent feature of the Transformer architecture: sufficiently long serial cognition must pass through the chain of thought (Korbak et al., 2025). We formalize this argument through the notion of opaque serial depth, given by the length of the longest computation that can be done without the use of interpretable intermediate steps like chain of thought. Given this formalization, we compute numeric upper bounds on the opaque serial depth of Gemma 3 models, as well as asymptotic results for additional architectures beyond standard LLMs. We also open-source an automated method that can calculate upper bounds on the opaque serial depth of arbitrary neural networks, and use it to demonstrate that Mixture-of-Experts models likely have lower depth than dense models. Overall, our results suggest that opaque serial depth is a useful tool for understanding the potential for models to do significant reasoning that is not externalized.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。