大模型在无意义符号中暗藏推理，可被直接读取。
Reading Between the Dots: Decoding Hidden Computation across Filler Tokens
- 通过隐藏状态逆向解码，还原模型内部推理过程。
- 对四类任务的中间结果还原准确率达80%-95%。
- 适合研究模型可解释性与安全监控的学者。
前沿大模型能在内容空白的填充符号（如点或计数序列）上进行多步推理，生成正确答案而无需可见思维链。这构成了行为监控的极限情况：表面符号不携带推理信息。但隐藏于输出之外，并不等于对我们不可见。在四类任务（事实检索、并行数值合成、字符串操作、上下文计算）中，两个开源权重的前沿模型（DeepSeek V3、Kimi K2）以结构化、可读方式对填充符号进行计算：注意力机制将问题经由填充区域导向答案，逻辑读出显示检索事实早期出现，组合过程在深层逐渐清晰；在填充位置移植键值缓存可因果性地交换不同示例的输出。我们提出一种仅需隐藏状态输入的无监督解码流程，在两个模型及所有四类任务上均实现80%-95%的中间值还原准确率（最佳大模型判别器），无需真实标签或训练。表明即使逃避了行为思维链监控的隐式计算，仍可通过残差流直接读取，提示可监控性是模型完整计算轨迹的属性，而非仅限于表面符号。
原文摘要 · Abstract (English)
Frontier LLMs can perform multi-step reasoning over content-free filler tokens like dots or counting sequences, producing correct answers with no visible chain-of-thought (CoT). This is a limit case for behavioral oversight, where surface tokens carry no information about the underlying reasoning. But hidden from the output is not the same as hidden from us. On four task families (fact retrieval, parallel numeric composition, string manipulation, and in-context computation), two open-weights frontier models (DeepSeek V3, Kimi K2) compute over filler tokens in a structured, legible way: attention routes the question through the filler region to the answer, logit-lens readouts show retrieved facts emerging early and their composition crystallizing in late layers, and KV-cache transplants at filler positions causally swap outputs between examples. We introduce an unsupervised decoding pipeline that takes only hidden states as input and recovers intermediate values with 80-95% accuracy (best LLM judge) across both models and all four tasks, without ground-truth labels or training. Hidden computation that defeats behavioral CoT monitoring is, on these tasks, directly readable from the residual stream, suggesting monitorability is a property of the model's full computational trace, not just its surface tokens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。