arXiv:2606.16196cs.LGcs.CV2026-06

通过语义扰动检测异常图像,让模型判断更透明可信。

When Confidence Lacks Concepts: Interpretable OOD Detection via Representation Perturbations

论文配图:When Confidence Lacks Concepts: Interpretable OOD Detection via Representation Perturbations
图 1 · 摘自论文原文
  • 用特定概念向量扰动深层特征,看预测是否稳定。
  • 真实数据扰动后预测变化小,异常数据变化大。
  • 适合医疗等高风险场景,结果可解释性强。

深度神经网络在医学影像任务中表现卓越,但在分布外样本上易过度泛化,威胁临床安全部署。现有分布外(OOD)检测方法多依赖难以理解的内部信号,降低可信度。本文提出一种可解释的OOD检测框架:基于稀疏自编码器(SAEs)从分布内数据学习类别特定的概念向量,将密集中间表示解耦为稀疏、语义明确的成分。推理时,用预测类别的概念向量扰动深层表示,并测量分类对数概率的稳定性。我们假设:分布内样本因特征与类别语义方向对齐,扰动后变化小;而分布外样本因表征错位,敏感度显著升高。该方法将OOD检测建模为条件概念稳定性分析,既提供判别性信号,又揭示模型不确定性的内在机制,特别适用于高风险医疗应用。

原文摘要 · Abstract (English)

Deep neural networks have achieved remarkable performance across medical imaging tasks, yet their tendency to overgeneralize under distributional shifts poses a major obstacle to safe clinical deployment. Out-of-Distribution (OOD) detection methods aim to mitigate this risk, but most existing approaches rely on opaque internal signals with poorly understood semantic meaning, limiting trust in safety-critical settings. In this work, we propose an interpretable OOD detection framework that probes the stability of model predictions under class-conditioned semantic perturbations. Leveraging sparse autoencoders (SAEs), we learn class-specific concept vectors from in-distribution data that disentangle dense intermediate representations into sparse, semantically meaningful components. At inference, we perturb deeper-layer representations using the concept vectors associated with the model's predicted class and measure the class logits stability. We hypothesize that in-distribution samples exhibit low sensitivity to such perturbations, as their representations align with class-specific semantic directions, whereas OOD samples show amplified deviations due to representational misalignment. By framing OOD detection as a concept conditioned stability analysis, our approach provides both a discriminative OOD signal and an interpretable lens into the internal mechanisms driving model uncertainty, making it particularly suitable for high stakes medical applications.

OOD检测可解释性医疗AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。