分离卷积网络中混杂概念的通道，让模型更易解释。
Disentangling Polysemantic Channels in Convolutional Neural Networks
- 通过前层激活模式差异重构权重，拆分混杂概念的通道
- 使每个通道只响应单一概念，提升特征可视化效果
- 适合研究模型可解释性或想理解神经网络决策机制的人
机制可解释性关注于分析卷积神经网络（CNN）中的个体组件及其如何构成代表决策机制的更大电路。这类研究具有挑战性，因为CNN常学习到编码多个概念的多义通道，难以解释。为此，我们提出一种算法，将特定类型的多义通道拆分为多个单义通道，每个通道仅响应一个概念。该方法通过重构CNN中的权重实现，利用同一通道内不同概念在前一层表现出不同的激活模式。通过拆分这些多义特征，提升了CNN的可解释性，最终改进了诸如特征可视化等解释技术。
原文摘要 · Abstract (English)
Mechanistic interpretability is concerned with analyzing individual components in a (convolutional) neural network (CNN) and how they form larger circuits representing decision mechanisms. These investigations are challenging since CNNs frequently learn polysemantic channels that encode distinct concepts, making them hard to interpret. To address this, we propose an algorithm to disentangle a specific kind of polysemantic channel into multiple channels, each responding to a single concept. Our approach restructures weights in a CNN, utilizing that different concepts within the same channel exhibit distinct activation patterns in the previous layer. By disentangling these polysemantic features, we enhance the interpretability of CNNs, ultimately improving explanatory techniques such as feature visualizations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。