大模型能提前预判自己还要生成多少字,且这种预判可被直接读取。
How Much is Left? LLMs Linearly Encode Their Remaining Output Length

- 用线性探测器从提示词末尾隐藏状态中直接预测总输出长度。
- 在七种数据集上预测准确率超统计基线,且跨数据集泛化能力强。
- 重写答案时模型内部估计值会突然上升,反映其有动态规划意识。
大语言模型逐词生成文本,但其输出长度结构高度一致:逐步解答收敛于可预测的词数,检索任务通常仅需几句话,修正内容则可量化解析。我们探究模型是否内含对剩余输出长度的估计。在三个7-80亿参数的开源模型上,对冻结的隐藏状态训练小型线性探测器,覆盖七种补全类数据集,发现三重证据:第一,仅凭提示词末尾隐藏状态即可线性解码总响应长度,无需任何输出;第二,自然语言数据集训练的探测方向具备广泛迁移能力,包括从未见过的受控合成补全,优于统计基线;反向则不成立,该不对称本身具信息量;第三,在精选高损失补全中,当模型重写部分解决方案时,探测器的每位置估计值会瞬间上升,这种方向性变化无法由仅依赖位置的预测器复现(定性而非聚合)。我们将其视为对剩余生成长度的近似估计,不同于变压器不可精确计数的理论限制,解释为大模型维持了类似计划的输出长度内部表征(可解码,未必因果使用)。
原文摘要 · Abstract (English)
Large language models generate one token at a time, yet their responses show remarkably consistent length structure: step-by-step solutions converge in predictable token counts, retrievals stop after a few sentences, retractions extend responses by measurable amounts. We ask whether the model carries an internal estimate of how much response remains. Training minimal-capacity linear probes on frozen hidden states of three open-weight 7-8B models across seven completion-style datasets, we find three converging pieces of evidence. First, total response length is linearly decodable from the prompt's last hidden state alone, before any output is emitted. Second, probe directions trained on natural-language datasets transfer broadly, including to controlled synthetic completions never seen in training, outperforming a statistical baseline; the converse direction generally fails, and this asymmetry is itself informative. Third, on curated high-loss completions, the probe's per-position estimate shifts upward at the moment the model retracts and restarts a partial solution, a directional behavior no position-only predictor can reproduce (qualitative, not aggregate). We frame this as approximate estimation of remaining generation length, distinct from exact-counting impossibility results for transformers, and interpret it as evidence that LLMs maintain a plan-like internal representation of output length (decodable, not necessarily used causally).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。