让深度模型更可解释,且不损失性能。
Interpretability Without Tradeoffs: Disentangling Polysemanticity At Equal Predictive Performance
- 通过重构信息流动方式分离混杂概念,无需重新训练
- 在DINOv2和ViT-B/16上保持原精度,可解释性显著提升
- 适用于预训练模型,适合需要透明决策的场景
深度神经网络广泛使用,但其学习内容难以解释。主要障碍是单个神经元常编码多个无关概念,掩盖了模型决策过程。以往方法如稀疏自编码器虽能将混合信号解耦为更清晰的‘单义’特征,但通常需修改模型,导致下游性能下降。为此,我们提出ELUDe(显式、无损、无监督解耦),可在不改变模型输出的前提下提升可解释性。ELUDe将潜在表示分解为清晰、可检查的子单元,行为类似可解释特征,且保证输出完全不变。它无需显式训练、无需标签,可应用于预训练模型。该方法通过重组层间信息流,重定向特定概念贡献,同时在构造上保留原始计算。在多个视觉模型(包括DINOv2和监督型ViT-B/16)上,ELUDe显著提升可解释性,维持下游准确率不变,运行高效,并支持表示操控等实际应用。总之,ELUDe实现了近乎无代价的可解释性:更清晰、可扩展、可操作的模型洞察,无性能损失。
原文摘要 · Abstract (English)
Deep neural networks (DNNs) are widely used, but interpreting what they actually learn remains difficult. A major obstacle is that individual neurons often encode multiple unrelated concepts, obscuring the decision process of the network. While prior work, such as sparse autoencoders, can separate these mixed signals into more meaningful, "monosemantic" features, this typically requires altering the model in ways that can degrade downstream performance. To overcome this, we introduce ELUDe (explicit, lossless, unsupervised disentanglement), a method for improving the interpretability of DNNs while preserving their functional equivalence. ELUDe breaks latent representations into clear, inspectable sub-units that behave like interpretable features, while guaranteeing that the model's outputs remain exactly the same. It requires no explicit training, no labels, and can be applied to pretrained models. ELUDe works by reorganizing how information flows between layers, re-routing concept-specific contributions while preserving the original computation by construction. Across several vision models, including DINOv2 and supervised ViT-B/16, ELUDe improves interpretability, keeps downstream accuracy unchanged, runs efficiently, and supports practical uses such as steering model representations. In short, ELUDe offers interpretability (almost) without a tradeoff: clearer, scalable, and actionable model insights with no loss in performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。