大模型虽只预测下一个词，却在隐藏状态中提前编码了整段回复的规划信息。
Emergent Response Planning in LLMs
- 通过探针检测发现，模型隐藏层已蕴含未来输出的结构、内容和行为特征。
- 模型越大，规划能力越强；生成过程中规划信息逐步显现。
- 为提升生成透明度与可控性提供了新思路，适合研究可解释AI的读者。
本文指出，尽管大型语言模型（LLMs）仅以预测下一个词为目标进行训练，但其隐藏表示中却涌现出前瞻性的规划行为：隐藏状态编码了超越下一个词的未来输出信息。通过简单的探针实验，我们证明了提示的隐藏表示包含了完整响应的全局属性，包括结构属性（如响应长度、推理步骤）、内容属性（如故事创作中的角色选择、结尾多选答案）以及行为属性（如回答置信度、事实一致性）。此外，我们探讨了该规划能力随模型规模的变化规律及生成过程中的演化特性。结果表明，大型语言模型在其隐藏表示中提前规划未来输出，这为提升生成过程的透明度和控制性提供了潜在应用价值。
原文摘要 · Abstract (English)
In this work, we argue that large language models (LLMs), though trained to predict only the next token, exhibit emergent planning behaviors: $\textbf{their hidden representations encode future outputs beyond the next token}$. Through simple probing, we demonstrate that LLM prompt representations encode global attributes of their entire responses, including $\textit{structure attributes}$ (e.g., response length, reasoning steps), $\textit{content attributes}$ (e.g., character choices in storywriting, multiple-choice answers at the end of response), and $\textit{behavior attributes}$ (e.g., answer confidence, factual consistency). In addition to identifying response planning, we explore how it scales with model size across tasks and how it evolves during generation. The findings that LLMs plan ahead for the future in their hidden representations suggest potential applications for improving transparency and generation control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。