让单领域模型更懂异常数据，提升真实场景下的可靠性。
Beyond the Class Subspace: Teacher-Guided Training for Reliable Out-of-Distribution Detection in Single-Domain Models
- 用预训练多域教师模型引导学生网络保留领域差异特征
- 在8个单域数据集上，远端异常检测误报率降低超10个百分点
- 无需额外计算开销，适合部署于资源受限的工业系统
现有分布外（OOD）检测方法在多领域基准上表现良好，但许多实际系统仅在单领域数据上训练。我们发现这种设置会引发几何失效模式——领域敏感性坍缩（DSC）：监督训练将特征压缩至低秩类别子空间，抑制了携带领域变化信号的方向。理论分析表明，在DSC下，基于距离和对数几率的OOD评分对领域偏移失去敏感性。为此，我们提出教师引导训练（TGT），通过冻结的多域教师模型（DINOv2）将被抑制的残差结构蒸馏到学生网络中。训练后丢弃教师与辅助头，不增加推理开销。在八个单域基准上，TGT使基于距离的评分器的远端OOD FPR@95显著下降：MDS提升11.61个百分点，ViM提升10.78个百分点，kNN提升12.87个百分点（以ResNet-50平均），同时保持或略微提升原域OOD检测与分类准确率。
原文摘要 · Abstract (English)
Out-of-distribution (OOD) detection methods perform well on multi-domain benchmarks, yet many practical systems are trained on single-domain data. We show that this regime induces a geometric failure mode, Domain-Sensitivity Collapse (DSC): supervised training compresses features into a low-rank class subspace and suppresses directions that carry domain-shift signal. We provide theory showing that, under DSC, distance- and logit-based OOD scores lose sensitivity to domain shift. We then introduce Teacher-Guided Training (TGT), which distills class-suppressed residual structure from a frozen multi-domain teacher (DINOv2) into the student during training. The teacher and auxiliary head are discarded after training, adding no inference overhead. Across eight single-domain benchmarks, TGT yields large far-OOD FPR@95 reductions for distance-based scorers: MDS improves by 11.61 pp, ViM by 10.78 pp, and kNN by 12.87 pp (ResNet-50 average), while maintaining or slightly improving in-domain OOD and classification accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。