用高效方法解析大模型内部状态,揭示预测演化路径。
Interpreting Language Model Hidden States at Scale

- 采用低秩变换与子集KL训练,降低参数和内存开销
- 在70B模型上实现482个镜头的密集集成,覆盖率达6倍提升
- 可跨组件分析,发现干预效果与行为显现位置不一致
Lens方法通过将中间激活映射到输出词表,揭示语言模型中下一个词预测的演化过程。传统训练镜头因参数量随模型宽度平方增长且需全词表KL训练,成本高昂,仅限于200亿参数以下模型,且局限于特定组件。本文提出OmniLens,统一适用于任意宽度模型的残差流、注意力层或MLP,并结合两项独立缩放技术:其一,低秩翻译器使每镜头参数增长线性于模型宽度,减少可训练参数达98.4%;其二,子集KL仅计算选定词元逻辑值:其Top-k模式可降低70%峰值训练内存,重要性采样变体则保持完整KL的无偏随机梯度。这些优势使我们能在LLaMA-3.3-70B上部署482个镜头的密集集合,模型级覆盖范围较单一残差流设计提升6倍。全面覆盖揭示了:行为最显著的组件未必是干预最有效的部位,且最有效干预位于先前研究未考察的注意力头之外。在三个案例研究(提示注入检测、多跳记忆注入、毒性定位)中,OmniLens以更低成本复现了已有关键结果。
原文摘要 · Abstract (English)
Lens methods interpret large language models (LLMs) by mapping intermediate activations to the output vocabulary, revealing how next-token predictions develop through the network. Trained lenses remain expensive: affine-translator parameters grow quadratically with model width, while exact, full-vocabulary Kullback--Leibler (KL) training dominates memory. Consequently, prior trained lenses have been applied to models of at most 20B parameters and remain tied to particular component types. We present OmniLens, which applies a single lens family to any model-width activation, whether residual stream, attention, or MLP, and combines two independent scaling techniques. First, low-rank translators make per-lens parameter growth linear in model width and reduce trainable parameters by up to 98.4%. Second, Subset-KL materializes only selected vocabulary logits: its Top-k mode cuts peak training memory by up to 70%, while its importance-sampled variant retains unbiased stochastic gradients for the full KL. These savings enable a dense ensemble of 482 lenses for LLaMA-3.3-70B, providing 6x the coverage of a residual-stream design at the same depth. Model-wide coverage then reveals what single-component lenses cannot: the components where a behavior is most visible need not be those where intervention is most effective, and the most effective interventions lie outside the attention heads examined by prior lens studies. Across three case studies (prompt-injection detection, multi-hop memory injection, and toxicity localization), OmniLens reproduces key published results at substantially lower cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。