将深度网络特征重新定义为局部专家的中心方向,提升可解释性。
The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts

- 用局部仿射专家的中心向量表示网络特征,替代传统激活方向。
- 在DINO ViT上得到更稀疏、下游任务更实用的特征词典。
- 统一解释工具为几何结构,实现机制化可解释性,适合模型分析者。
线性表示假设(LRH)将深度网络(DN)的特征视为激活空间中的线性方向,即中间层的输出空间。该表征使网络的输入-输出映射与特征方向的组织解耦。本文提出线性中心假设(LCH),将特征识别为网络中心空间中的线性方向——其中任意向量代表一个局部仿射专家的中心,精确刻画了网络的输入-输出映射(如分段仿射网络)或近似刻画(如Transformer等平滑网络)。我们证明,用中心向量替换中间激活,可作为标准可解释性工具的功能替代方案。实验表明,此方法在DINO ViTs上产生更稀疏、更适用于下游任务的特征词典,在控制任务中抑制虚假方向,恢复GPT2-Large中可解释的神经回路,并生成忠实的梯度显著性图。LCH将特征词典、探针、神经回路和显著性图统一为一个基于网络输入-输出映射的几何对象,使可解释性从构造上即具备机制性。代码详见:https://github.com/ThomasWalker1/LinearCentroidsHypothesis。
原文摘要 · Abstract (English)
The Linear Representation Hypothesis (LRH) identifies features of a trained deep network (DN) as linear directions in the activation spaces, i.e., output spaces of intermediate layers. This characterization decouples the input-output maps learned by a DN from the organization of feature directions in its activation spaces. We introduce the Linear Centroids Hypothesis (LCH), which instead identifies features with linear directions among a DN's centroid spaces -- where any vector denotes a centroid or summary of a local affine expert characterizing the learned input-output maps of the DN exactly (e.g., for piecewise-affine DNs) or approximately (e.g., for smooth DNs like transformers). We show that replacing intermediate activations with centroids yields a functional drop-in alternative for standard interpretability tools. Empirically, this change yields sparser, more downstream-useful feature dictionaries on DINO ViTs, suppresses spurious directions on a controlled task, recovers interpretable circuits in GPT2-Large, and produces faithful gradient-based saliency maps. LCH unifies dictionaries, probing, circuits, and saliency maps into a single geometric object grounded in the network's input-output map -- making interpretability mechanistic by construction rather than post hoc. Code to study the LCH https://github.com/ThomasWalker1/LinearCentroidsHypothesis .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。