通过修复单个令牌激活，发现大模型能提前规划段落内容。
Extracting Paragraphs from LLM Token Activations
- 用换行符令牌激活的修复方法研究段落生成逻辑
- 修复单个令牌可传递后续段落的关键上下文信息
- 适合关注模型内部工作机制的研究者
生成式大语言模型在自然语言处理任务中表现优异，但其内部机制仍远未被充分理解，尤其在词元级别预测之外。本研究探讨了这些模型是否在段落开始时就已决定内容，从而揭示其上下文理解能力。通过分析单个词元激活中的信息，特别是双换行符令牌（\n\n）的激活，我们证明：修复这些激活能够传递关于后续段落的重要上下文信息，进一步说明模型具备前瞻规划能力。
原文摘要 · Abstract (English)
Generative large language models (LLMs) excel in natural language processing tasks, yet their inner workings remain underexplored beyond token-level predictions. This study investigates the degree to which these models decide the content of a paragraph at its onset, shedding light on their contextual understanding. By examining the information encoded in single-token activations, specifically the "\textbackslash n\textbackslash n" double newline token, we demonstrate that patching these activations can transfer significant information about the context of the following paragraph, providing further insights into the model's capacity to plan ahead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。