解析代码大模型如何补全括号，揭示其内部决策机制。
Mechanistic Understanding of Language Models in Syntactic Code Completion
- 通过机械可解释性分析，研究代码模型在括号补全任务中的内部运作。
- 中间到后期层才具备准确预测闭合括号的能力，注意力机制起关键作用。
- 发现部分注意力头能精确追踪已关闭括号数量，影响模型最终表现。
近期语言模型（LMs）在代码生成任务中展现出惊人能力，尤其在针对代码数据集微调后，这类模型被称为代码语言模型（Code LMs）。然而，我们对这些模型内部决策过程的理解仍十分有限，例如它们如何运用语法或语义知识，这可能在实际应用中导致意外风险。为此，本文开展了一项早期的机械可解释性研究，聚焦于理解代码大模型（CodeLlama-7b）在括号补全任务中的表现。研究发现，模型需依赖中间至后期层才能稳定预测正确闭合括号；多头注意力（MHA）和前馈（FF）子层均起重要作用，其中MHA尤为关键；此外，某些注意力头能精确跟踪已关闭括号数量，但未必促进缺失括号的正确补全，可能正向或负向影响性能。
原文摘要 · Abstract (English)
Recently, language models (LMs) have shown impressive proficiency in code generation tasks, especially when fine-tuned on code-specific datasets, commonly known as Code LMs. However, our understanding of the internal decision-making processes of Code LMs, such as how they use their (syntactic or semantic) knowledge, remains limited, which could lead to unintended harm as they are increasingly used in real life. This motivates us to conduct one of the first Mechanistic Interpretability works to understand how Code LMs perform a syntactic completion task, specifically the closing parenthesis task, on the CodeLlama-7b model (Roziere et al. 2023). Our findings reveal that the model requires middle-later layers until it can confidently predict the correct label for the closing parenthesis task. Additionally, we identify that while both multi-head attention (MHA) and feed-forward (FF) sub-layers play essential roles, MHA is particularly crucial. Furthermore, we also discover attention heads that keep track of the number of already closed parentheses precisely but may or may not promote a correct number of closing parentheses that are still missing, leading to a positive or negative impact on the model's performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。