提出三类逻辑门,实现语言模型电路的完整可解释性
Rethinking Circuit Completeness in Language Models: AND, OR, and ADDER Gates
- 用AND、OR、ADDER三类逻辑门重构电路结构
- 新框架使电路识别更完整且稳定,不遗漏关键机制
- 适合研究模型可解释性的研究人员使用
电路发现已成为机械可解释性的重要方法,而电路完整性问题日益受到关注。现有方法因无法保证完整性,导致结果在不同运行间不一致,并遗漏关键机制,其根源在于对OR门的检测不充分。本文系统引入AND、OR、ADDER三种逻辑门,将电路分解为这些基本单元,推导出实现忠实性与完整性的最小条件。提出一种结合噪声注入与去噪干预的框架,可无缝集成至现有方法中,计算开销小,能完全识别并区分电路中的逻辑门。实验验证了该框架在恢复电路忠实性、完整性与稀疏性方面的有效性,并揭示了三类门的比例及其对输出的贡献,深入分析了它们在语言模型功能中的表现。
原文摘要 · Abstract (English)
Circuit discovery has gradually become one of the prominent methods for mechanistic interpretability, and research on circuit completeness has also garnered increasing attention. Methods of circuit discovery that do not guarantee completeness not only result in circuits that are not fixed across different runs but also cause key mechanisms to be omitted. The nature of incompleteness arises from the presence of OR gates within the circuit, which are often only partially detected in standard circuit discovery methods. To this end, we systematically introduce three types of logic gates: AND, OR, and ADDER gates, and decompose the circuit into combinations of these logical gates. Through the concept of these gates, we derive the minimum requirements necessary to achieve faithfulness and completeness. Furthermore, we propose a framework that combines noising-based and denoising-based interventions, which can be easily integrated into existing circuit discovery methods without significantly increasing computational complexity. This framework is capable of fully identifying the logic gates and distinguishing them within the circuit. In addition to the extensive experimental validation of the framework's ability to restore the faithfulness, completeness, and sparsity of circuits, using this framework, we uncover fundamental properties of the three logic gates, such as their proportions and contributions to the output, and explore how they behave among the functionalities of language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。