arXiv:2604.16426cs.LG2026-04

通过分析神经元激活区域,构建稳定的函数相似性度量。

Functional Similarity Metric for Neural Networks: Overcoming Parametric Ambiguity via Activation Region Analysis

  • 基于激活区域拓扑构造神经网络的规范表示。
  • 对微小权重扰动仍保持稳定,有效缓解神经元闪烁现象。
  • 适合模型融合、剪枝评估和可解释性研究场景。

随着深度学习模型复杂度提升,表征歧义成为影响可解释性和可靠合并的关键障碍。对于ReLU网络,相同的函数映射可通过完全不同的权重配置实现,源于神经元置换和正对角缩放等代数对称性。传统基于参数的比较方法在训练中对微小权重扰动极度敏感。本文提出一种数学严谨的方法,构建神经网络的稳定规范表示及鲁棒的函数相似性度量。我们从比较原始权重转向分析神经元激活区域的拓扑结构。算法首先通过权向量L2归一化并后续层补偿消除缩放歧义;接着在数据样本上生成激活区域的离散二值近似,作为功能签名;为克服大二值向量比较的计算瓶颈,采用局部敏感哈希(MinHash)快速近似计算杰卡德指数;最终将跨网络神经元匹配建模为线性和分配问题,用匈牙利算法求解。理论与实验均证明,该度量能有效抑制神经元“闪烁”效应,对微小权重扰动具有极强鲁棒性。该框架为模型合并、迁移学习、剪枝过程的客观评估以及可解释人工智能提供了坚实基础。

原文摘要 · Abstract (English)

As modern deep learning architectures grow in complexity, representational ambiguity emerges as a critical barrier to their interpretability and reliable merging. For ReLU networks, identical functional mappings can be achieved through entirely different weight configurations due to algebraic symmetries: neuron permutation and positive diagonal scaling. Consequently, traditional parameter-based comparison methods exhibit extreme instability to slight weight perturbations during training. This paper proposes a mathematically grounded approach to constructing a stable canonical representation of neural networks and a robust functional similarity metric. We shift focus from comparing raw weights to analyzing the topology of neuron activation regions. The algorithm first eliminates scaling ambiguity via L2-normalization of weight vectors with subsequent layer compensation. Next, discrete approximations of activation regions are generated as binary functional signatures evaluated over a data sample. To overcome the computational bottleneck of comparing large binary vectors, we adapt Locality-Sensitive Hashing, specifically MinHash, providing a fast and statistically precise approximation of the Jaccard index. The final cross-network neuron matching is formulated as a linear sum assignment problem solved via the Hungarian algorithm. We demonstrate theoretically and experimentally that our metric mitigates the neuron "flickering" effect and exhibits exceptional robustness to minor weight perturbations. This framework provides a solid foundation for model merging, transfer learning, objective assessment during pruning, and Explainable AI paradigms.

神经网络相似性度量可解释性模型融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。