让大模型自解释内部计算，效果比外人解释更好。
Training Language Models to Explain Their Own Computations
- 用自身内部信息训练模型生成自然语言解释
- 仅需数万样本即实现对新任务的有效泛化
- 自解释优于他人解释，尤其适合高阶模型
大模型能否学会忠实描述其内部计算过程？它们是否比其他模型更擅长自解释？我们研究了利用大模型对自身内部状态的特权访问，来生成行为解释的新方法。基于现有可解释性技术作为真实标签，我们微调大模型，使其生成自然语言描述：（1）特征编码的信息内容，（2）内部激活的因果结构，（3）特定输入词元对输出的影响。仅用数万条示例进行训练，解释模型即可在新查询上展现非平凡的泛化能力。该泛化部分归因于模型对自身内部状态的特权访问——用同一模型解释自身计算，通常优于用更强大但不同的模型去解释目标模型（即使后者更强）。结果表明，大模型不仅能可靠地解释自身计算，且此类解释可成为现有可解释性方法的可扩展补充。代码与数据见 https://github.com/TransluceAI/introspective-interp
原文摘要 · Abstract (English)
Can language models (LMs) learn to faithfully describe their internal computations? Are they better able to describe themselves than other models? We study the extent to which LMs' privileged access to their own internals can be leveraged to produce new techniques for explaining their behavior. Using existing interpretability techniques as a source of ground truth, we fine-tune LMs to generate natural language descriptions of (1) the information encoded by LM features, (2) the causal structure of LMs' internal activations, and (3) the influence of specific input tokens on LM outputs. When trained with only tens of thousands of example explanations, explainer models exhibit non-trivial generalization to new queries. This generalization appears partly attributable to explainer models' privileged access to their own internals: using a model to explain its own computations generally works better than using a *different* model to explain its computations (even if the explainer model is significantly more capable than the target). Our results suggest not only that LMs can learn to reliably explain their internal computations, but that such explanations offer a scalable complement to existing interpretability methods. Code and data at https://github.com/TransluceAI/introspective-interp
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。