用对比样本提升神经元标注精度,让解释更准确具体。
Contrastive Semantic Projection: Faithful Neuron Labeling with Contrastive Examples

- 用视觉语言模型结合对比图像生成候选标签
- 引入对比语义投影(CSP)优化标签选择,提升准确性
- 适用于医学图像等需要高精度解释的场景
神经元标注旨在为深度网络内部单元赋予文本描述。现有方法多依赖高度激活样本,常因关注主导但偶然的视觉特征而产生宽泛或误导性标签。此前工作如FALCON引入语义相似但激活低的对比样本以增强解释,但主要解决子空间级可解释性,难以扩展至神经元级标注。本文在两个阶段重访对比解释:(1) 利用视觉语言模型(VLMs)基于对比图像集生成候选标签;(2) 通过类CLIP编码器进行标签分配。实验表明,向VLM提供对比图像集可生成更具体、更忠实的标签。进一步提出对比语义投影(CSP),在SemanticLens基础上将对比样本直接融入其基于CLIP的评分与选择流程。在多项实验及黑色素瘤检测案例研究中,对比标注显著优于当前最优基线,在忠实度与语义粒度上均有提升。结果表明,对比样本是神经元标注与分析中简单却强大且未被充分挖掘的关键组件。
原文摘要 · Abstract (English)
Neuron labeling assigns textual descriptions to internal units of deep networks. Existing approaches typically rely on highly activating examples, often yielding broad or misleading labels by focusing on dominant but incidental visual factors. Prior work such as FALCON introduced contrastive examples -- inputs that are semantically similar to activating examples but elicit low activations -- to sharpen explanations, but it primarily addresses subspace-level interpretability rather than scalable neuron-level labeling. We revisit contrastive explanations for neuron-level labeling in two stages: (1) candidate label generation with vision language models (VLMs) and (2) label assignment with CLIP-like encoders. First, we show that providing contrastive image sets to VLMs yields candidate labels that are more specific and more faithful. Second, we introduce Contrastive Semantic Projection (CSP), an extension of SemanticLens that incorporates contrastive examples directly into its CLIP-based scoring and selection pipeline. Across extensive experiments and a case study on melanoma detection, contrastive labeling improves both faithfulness and semantic granularity over state-of-the-art baselines. Our results demonstrate that contrastive examples are a simple yet powerful and currently underutilized component of neuron labeling and analysis pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。