在对数空间中用锚点建模,精准估计模型在分布外数据上的准确率。
ALSA: Anchors in Logit Space for Out-of-Distribution Accuracy Estimation
- 直接在对数空间操作,避免概率压缩导致的信息损失。
- 通过可学习的锚点与影响函数,捕捉对数分布特征,预测准确率更准。
- 适用于视觉、语言、图等多种任务,尤其在分布外场景表现稳健。
在真实世界机器学习应用中,估计模型在未见无标签数据上的准确率至关重要,尤其当分布偏移导致性能下降时。现有方法通常依赖预测类别概率(softmax分数)或数据相似性度量。尽管基于softmax的方法在标准单纯形上表示预测有优势,但将对数压缩为概率会引发信息丢失;而相似性方法计算成本高且领域依赖性强,适用范围有限。本文提出ALSA(对数空间中的锚点用于准确率估计),一种新框架,通过直接在对数空间操作保留更丰富信息。基于理论洞察与实证观察,我们发现对数的聚合与分布与模型预测性能高度相关。为此,ALSA采用锚点建模策略:在对数空间初始化多个可学习锚点,每个分配一个影响函数以捕捉对数的细微变化。该方法在多种视觉、语言和图基准测试中显著优于基于softmax与相似性的基线,尤其在显著分布偏移下仍保持鲁棒性,展现出作为可靠模型评估工具的潜力。
原文摘要 · Abstract (English)
Estimating model accuracy on unseen, unlabeled datasets is crucial for real-world machine learning applications, especially under distribution shifts that can degrade performance. Existing methods often rely on predicted class probabilities (softmax scores) or data similarity metrics. While softmax-based approaches benefit from representing predictions on the standard simplex, compressing logits into probabilities leads to information loss. Meanwhile, similarity-based methods can be computationally expensive and domain-specific, limiting their broader applicability. In this paper, we introduce ALSA (Anchors in Logit Space for Accuracy estimation), a novel framework that preserves richer information by operating directly in the logit space. Building on theoretical insights and empirical observations, we demonstrate that the aggregation and distribution of logits exhibit a strong correlation with the predictive performance of the model. To exploit this property, ALSA employs an anchor-based modeling strategy: multiple learnable anchors are initialized in logit space, each assigned an influence function that captures subtle variations in the logits. This allows ALSA to provide robust and accurate performance estimates across a wide range of distribution shifts. Extensive experiments on vision, language, and graph benchmarks demonstrate ALSA's superiority over both softmax- and similarity-based baselines. Notably, ALSA's robustness under significant distribution shifts highlights its potential as a practical tool for reliable model evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。