语言模型生成括号时出错，因错误机制盖过正确机制。
Failure by Interference: Language Models Make Balanced Parentheses Errors When Faulty Mechanisms Overshadow Sound Ones
- 通过分析注意力头和前馈神经元，发现模型存在可靠与不可靠两种机制。
- 使用新方法RASteer可将括号平衡任务准确率从0%提升至约100%。
- 该方法适用于算术推理等任务，性能最高提升约20%，适合模型优化研究者。
尽管语言模型（LMs）在编程能力上取得显著进展，但在生成平衡括号等简单语法任务上仍表现不佳。本研究针对不同规模的模型（124M-7B）探究此类错误的内在机制。研究发现，模型依赖多个组件（注意力头和前馈神经元）独立做出预测：部分组件在多种输入下稳定输出正确结果（即“可靠机制”），而另一些则引入噪声，倾向于输出错误标记（即“故障机制”）。当故障机制主导预测时，错误便发生。基于此，我们提出RASteer，一种系统识别并增强可靠组件贡献的控制方法。该方法显著提升模型在平衡括号任务上的表现，使部分模型准确率从0%跃升至约100%，且未损害其通用编程能力。进一步验证表明，该方法在算术推理任务中亦具广泛适用性，性能最高提升约20%。
原文摘要 · Abstract (English)
Despite remarkable advances in coding capabilities, language models (LMs) still struggle with simple syntactic tasks such as generating balanced parentheses. In this study, we investigate the underlying mechanisms behind the persistence of these errors across LMs of varying sizes (124M-7B) to both understand and mitigate the errors. Our study reveals that LMs rely on a number of components (attention heads and FF neurons) that independently make their own predictions. While some components reliably promote correct answers across a generalized range of inputs (i.e., implementing "sound mechanisms''), others are less reliable and introduce noise by promoting incorrect tokens (i.e., implementing "faulty mechanisms''). Errors occur when the faulty mechanisms overshadow the sound ones and dominantly affect the predictions. Motivated by this insight, we introduce RASteer, a steering method to systematically identify and increase the contribution of reliable components for improving model performance. RASteer substantially improves performance on balanced parentheses tasks, boosting accuracy of some models from $0$% to around $100$% without impairing the models' general coding ability. We further demonstrate its broader applicability in arithmetic reasoning tasks, achieving performance gains of up to around $20$%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。