将符号算法注入预训练模型，提升其逻辑推理能力
Tracr-Injection: Distilling Algorithms into Pre-trained Language Models
- 用RASP语言编写的算法直接注入预训练模型权重
- 在模型残差流中形成可解读的符号变量空间
- 显著提升模型在分布外任务上的表现，适合研究符号推理
受大型语言模型兴起的推动，学界试图形式化表征Transformer架构内在的符号能力。为此提出了编程语言RASP，可直接编译为Transformer权重以实现特定算法。然而，RASP能实现的任务通常难以从自然无监督数据中学习，暴露出理论能力与实际可学习性之间的差距。本文提出tracr-injection方法，可将RASP编写的算法直接注入预训练语言模型。我们通过注入3个不同算法验证该方法，发现其在模型残差流中创建了可解释的子空间，该空间可解码为RASP代码中的变量。此外，相比基线模型，该方法显著提升了模型在分布外任务上的性能，表明模型内部确实发生了更符号化的机制运作。实验代码已开源。
原文摘要 · Abstract (English)
Motivated by the surge of large language models, there has been a push to formally characterize the symbolic abilities intrinsic to the transformer architecture. A programming language, called RASP, has been proposed, which can be directly compiled into transformer weights to implement these algorithms. However, the tasks that can be implemented in RASP are often uncommon to learn from natural unsupervised data, showing a mismatch between theoretical capabilities of the transformer architecture, and the practical learnability of these capabilities from unsupervised data. We propose tracr-injection, a method that allows us to distill algorithms written in RASP directly into a pre-trained language model. We showcase our method by injecting 3 different algorithms into a language model. We show how our method creates an interpretable subspace within the model's residual stream, which can be decoded into the variables present in the code of the RASP algorithm. Additionally, we found that the proposed method can improve out-of-distribution performance compared to our baseline, indicating that indeed a more symbolic mechanism is taking place in the inner workings of the model. We release the code used to run our experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。