通过激活修补定位大模型开始执行指令的层级位置。
Where does an LLM begin computing an instruction?
- 用三组简单数据集和层间激活修补法，追踪指令执行起点。
- 发现所有Llama模型在约第12层出现关键转折点，此前干预有效，此后无效。
- 方法可复现，适合研究模型内部行为与不同任务/规模的对比。
指令执行包含读取内容、理解指令、执行和生成答案等子过程。本文探究指令执行从‘读’转向‘做’的具体层级位置。设计了三组基础数据集（键值对、引文归属、字母选择）及两组多跳组合任务。基于最小差异提示对，在不同层进行激活修补，测量层间预测变化率。在多个Llama系列模型中，均观察到一个显著转折点，称为‘ onset’——在此前替换残差激活能改变输出，而在此后则基本无效。多跳任务也呈现相似的 onset 位置。该方法提供了一种简洁、可重复的手段，用于定位指令遵循的起始层级，并支持跨任务与模型规模的比较。
原文摘要 · Abstract (English)
Following an instruction involves distinct sub-processes, such as reading content, reading the instruction, executing it, and producing an answer. We ask where, along the layer stack, instruction following begins, the point where reading gives way to doing. We introduce three simple datasets (Key-Value, Quote Attribution, Letter Selection) and two hop compositions of these tasks. Using activation patching on minimal-contrast prompt pairs, we measure a layer-wise flip rate that indicates when substituting selected residual activations changes the predicted answer. Across models in the Llama family, we observe an inflection point, which we term onset, where interventions that change predictions before this point become largely ineffective afterward. Multi-hop compositions show a similar onset location. These results provide a simple, replicable way to locate where instruction following begins and to compare this location across tasks and model sizes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。