通过干预特征方向提升模型对异常数据的识别能力。
Gradient Short-Circuit: Efficient Out-of-Distribution Detection via Feature Intervention
- 利用输入邻域梯度一致性差异,识别异常样本
- 仅需一次前向传播即可实现高效检测,准确率显著提升
- 无需修改训练流程,适合部署在实际系统中
在开放世界环境中,深度模型需具备识别分布外(OOD)输入的能力。我们发现:在分布内(ID)样本附近,增强其预测类别的局部梯度方向保持一致;而分布外(OOD)样本则表现出混乱或矛盾的梯度方向。基于此,提出一种推理阶段的特征干预方法,短路被异常梯度利用以虚增置信度的特征坐标,同时保留对分布内样本的分类性能。为避免重新计算输出带来的开销,引入局部一阶近似,无需第二次前向传播即可准确估计修改后的结果。在标准OOD基准测试上,本方法取得显著性能提升。该方法轻量且仅需极少改动标准推理流程,为真实场景中的鲁棒性检测提供实用路径。
原文摘要 · Abstract (English)
Out-of-Distribution (OOD) detection is critical for safely deploying deep models in open-world environments, where inputs may lie outside the training distribution. During inference on a model trained exclusively with In-Distribution (ID) data, we observe a salient gradient phenomenon: around an ID sample, the local gradient directions for "enhancing" that sample's predicted class remain relatively consistent, whereas OOD samples--unseen in training--exhibit disorganized or conflicting gradient directions in the same neighborhood. Motivated by this observation, we propose an inference-stage technique to short-circuit those feature coordinates that spurious gradients exploit to inflate OOD confidence, while leaving ID classification largely intact. To circumvent the expense of recomputing the logits after this gradient short-circuit, we further introduce a local first-order approximation that accurately captures the post-modification outputs without a second forward pass. Experiments on standard OOD benchmarks show our approach yields substantial improvements. Moreover, the method is lightweight and requires minimal changes to the standard inference pipeline, offering a practical path toward robust OOD detection in real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。