用拓扑几何约束神经网络特征,让解释性更强
Learning Coherent Representations: A Topological Approach to Interpretability

- 引入'一致性'概念,要求特征与样本在几何上连续关联
- 实验证明可使特征空间具有清晰的拓扑结构,如圆形数据对应连续弧段
- 适合追求模型可解释性的研究者,尤其在生成与表示学习场景
深度神经网络学到的表征中,单个特征往往缺乏可解释性;一个神经元可能对分散且无关的输入激活。本文提出‘一致性’这一受大脑神经编码启发的几何属性:网格细胞和头方向细胞对状态空间中连续区域响应。若非负矩阵中每行(样本)只关注几何聚集的列(特征),反之亦然,且每个样本由某些特征充分描述、每个特征又服务于某些样本,则称其为一致矩阵。我们证明,一致矩阵诱导样本与特征的维托里斯-瑞普斯滤波之间存在有界交错,确保两者共享兼容的拓扑结构。该几何约束促进可解释性:例如,当数据位于圆环上时,一致特征必须将其划分为连续弧段。我们提出Coh——一种基于弗雷歇方差的可微目标函数,在训练中施加一致性。不同于仅限制激活数量的稀疏性,一致性要求激活样本在几何上连通,而非仅稀少。这不仅产生可解释特征,还构建出可解释的特征空间。我们在自编码器中使用合成数据和旋转后的MNIST数据进行验证,并在BERT的词嵌入中应用语言数据,结果表明该方法有效提升可解释性。
原文摘要 · Abstract (English)
Deep neural networks learn representations where individual features often lack interpretable meaning; a single neuron may activate for scattered, unrelated inputs. We introduce coherence, a geometric property inspired by neural coding in the brain, where neurons like grid cells and head direction cells respond to contiguous regions of state space. A non-negative matrix is coherent if each row (sample) attends to geometrically clustered columns (features) and vice versa, and in addition every sample is well described by some feature and every feature is needed by some sample. We prove that coherent matrices induce a bounded interleaving between the Vietoris-Rips filtrations of samples and features, guaranteeing that both spaces share compatible topological structure. This geometric constraint facilitates interpretability. For example, if data lies on a circle, coherent features must tile that circle into contiguous arcs. We introduce Coh, a differentiable objective function based on Fréchet variance that enforces coherence during training. Unlike sparsity, which bounds how many samples a feature activates on, coherence bounds which samples, requiring geometric connectivity rather than only rarity. This yields not just interpretable features but an interpretable feature space. We validate Coh in an auto-encoder using synthetic and rotated MNIST datasets and in a token embedding of BERT using language data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。