提出张量相似度，可精准判断神经网络模块是否功能等价。
When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability
- 基于张量的权重度量，无视参数空间对称性干扰。
- 能捕捉跨层机制，追踪训练过程中的功能演化。
- 适合研究模型可解释性与后门检测的科研人员。
机制可解释性旨在将模型分解为有意义的组件；验证两个组件是否实现相同计算是前提。现有相似性度量要么仅评估经验行为，对分布外机制无感；要么依赖基底相关的参数，忽略权重空间对称性。针对张量化模型，本文提出一种权重基础的度量——张量相似度，其对这些对称性不变。该度量能捕捉全局功能等价性，并通过高效递归算法考虑跨层机制。实验表明，张量相似度比现有方法更精确地跟踪功能训练动态，如涌现学习（grokking）和后门植入。这使相似性测量与忠实性验证转化为可解的代数问题，而非经验近似难题。
原文摘要 · Abstract (English)
Mechanistic interpretability aims to break models into meaningful parts; verifying that two such parts implement the same computation is a prerequisite. Existing similarity measures evaluate either empirical behaviour, leaving them blind to out-of-distribution mechanisms, or basis-dependent parameters, meaning they disregard weight-space symmetries. To address these issues for the class of tensor-based models, we introduce a weight-based metric, tensor similarity, that is invariant to such symmetries. This metric captures global functional equivalence and accounts for cross-layer mechanisms using an efficient recursive algorithm. Empirically, tensor similarity tracks functional training dynamics, such as grokking and backdoor insertion, with higher fidelity than existing metrics. This reduces measuring similarity and verifying faithfulness into a solved algebraic problem rather than one of empirical approximation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。