通过动态调整温度参数,提升开放集识别的泛化能力。
Boosting Open Set Recognition Performance through Modulated Representation Learning
- 设计可调温度调度策略,逐步优化特征表示。
- 在多个基准上显著提升开放集与闭集识别性能。
- 无需额外计算开销,兼容主流损失函数,适合实际部署。
开放集识别(OSR)旨在识别训练中未出现的新类别样本,对实际应用至关重要。现有方法使用固定温度缩放logits,限制了模型在实例级到语义级特征间的探索。本文提出温度调节表示学习方法,引入新型负余弦调度等温度策略。该策略使模型初期聚焦少数邻居形成粗略决策边界,随训练逐步扩大邻域以平滑边界,从而构建更丰富、更具泛化性的表示空间。相比需额外正则化或负样本的其他方法,本方案可无缝融入任意现有OSR损失函数,无计算开销。在交叉熵、对比学习及ARPL等多种基线上的实验表明,该方法普遍提升OSR与闭集性能,尤其在更具挑战性的语义迁移基准上表现优异。项目代码将公开。
原文摘要 · Abstract (English)
The open set recognition (OSR) problem aims to identify test samples from novel semantic classes that are not part of the training classes, a task that is crucial in many practical scenarios. However, the existing OSR methods use a constant scaling factor (the temperature) to the logits before applying a loss function, which hinders the model from exploring both ends of the spectrum in representation learning -- from instance-level to semantic-level features. In this paper, we address this problem by enabling temperature-modulated representation learning using a set of proposed temperature schedules, including our novel negative cosine schedule. Our temperature schedules allow the model to form a coarse decision boundary at the beginning of training by focusing on fewer neighbors, and gradually prioritizes more neighbors to smooth out the rough edges. This gradual task switching leads to a richer and more generalizable representation space. While other OSR methods benefit by including regularization or auxiliary negative samples, such as with mix-up, thereby adding a significant computational overhead, our schedules can be folded into any existing OSR loss function with no overhead. We implement the novel schedule on top of a number of baselines, using cross-entropy, contrastive and the ARPL loss functions and find that it boosts both the OSR and the closed set performance in most cases, especially on the tougher semantic shift benchmarks. Project codes will be available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。