用潜在空间对比嵌入引导模型关注重要特征,提升分类性能。
SAGE: Saliency-Guided Contrastive Embeddings
- 在模型潜空间中通过对比损失引导关注显著特征
- 在开集和闭集任务中均超越现有最优方法
- 适合需要提升模型可解释性与泛化能力的高风险场景
将人类感知先验融入神经网络训练已被证明能提升模型泛化能力、作为有效正则化手段,并使模型在高风险领域更贴近人类专家判断。现有基于显著性的训练方法多依赖模型内部机制,但近期研究指出其可靠性存疑。我们发现,显著性引导训练的诸多挑战源于将引导机制局限于图像空间。因此,我们转而使用模型的潜在空间嵌入,在训练中引导人类显著性信息,提出SAGE(Saliency-Guided Contrastive Embeddings):一种利用对比嵌入整合人类显著性的损失函数。对输入施加保持显著性与破坏显著性的信号增强,捕捉嵌入与模型输出概率的变化,通过对比三元组损失引导模型关注显著特征,避开非显著特征。同时,通过检查输出概率分布进行合理性验证。实验表明,相比当前最优的显著性引导方法,SAGE在开集与闭集场景下均有性能提升,适用于多种骨干网络,并展现出跨任务的广泛泛化能力。
原文摘要 · Abstract (English)
Integrating human perceptual priors into the training of neural networks has been shown to raise model generalization, serve as an effective regularizer, and align models with human expertise for applications in high-risk domains. Existing approaches to integrate saliency into model training often rely on internal model mechanisms, which recent research suggests may be unreliable. Our insight is that many challenges associated with saliency-guided training stem from the placement of the guidance approaches solely within the image space. Instead, we move away from the image space, use the model's latent space embeddings to steer human guidance during training, and we propose SAGE (Saliency-Guided Contrastive Embeddings): a loss function that integrates human saliency into network training using contrastive embeddings. We apply salient-preserving and saliency-degrading signal augmentations to the input and capture the changes in embeddings and model logits. We guide the model towards salient features and away from non-salient features using a contrastive triplet loss. Additionally, we perform a sanity check on the logit distributions to ensure that the model outputs match the saliency-based augmentations. We demonstrate a boost in classification performance across both open- and closed-set scenarios against SOTA saliency-based methods, showing SAGE's effectiveness across various backbones, and include experiments to suggest its wide generalization across tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。