让语言模型自己解释微调带来的权重变化。
Learning to Interpret Weight Differences in Language Models
- 用合成数据训练模型自述微调后的改变。
- 能准确描述微调引发的隐藏行为和知识更新。
- 适合研究模型内部机制或可解释性的人看。
微调（finetuning）是更新预训练语言模型参数知识、使其适应新任务和领域的标准方法。然而，微调引起的模型权重变化（权重差异，weight diffs）通常难以解释。虽然查看微调数据集可大致了解模型变化，但这些数据集常不公开或规模过大难以直接使用。为此，我们提出差分解释微调（Diff Interpretation Tuning, DIT），一种训练模型自我描述其微调引发修改的方法。该方法利用合成的带标签权重差异数据，训练一个DIT-adapter，可应用于兼容的微调模型，使其生成自然语言描述自身变化的内容。我们在两个概念验证场景（报告隐藏行为、总结微调知识）中证明，该方法能使模型以准确的自然语言描述其微调导致的修改。
原文摘要 · Abstract (English)
Finetuning (pretrained) language models is a standard approach for updating their internal parametric knowledge and specializing them to new tasks and domains. However, the corresponding model weight changes ("weight diffs") are not generally interpretable. While inspecting the finetuning dataset can give a sense of how the model might have changed, these datasets are often not publicly available or are too large to work with directly. Towards the goal of comprehensively understanding weight diffs in natural language, we introduce Diff Interpretation Tuning (DIT), a method that trains models to describe their own finetuning-induced modifications. Our approach uses synthetic, labeled weight diffs to train a DIT-adapter, which can be applied to a compatible finetuned model to make it describe how it has changed. We demonstrate in two proof-of-concept settings (reporting hidden behaviors and summarizing finetuned knowledge) that our method enables models to describe their finetuning-induced modifications using accurate natural language descriptions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。