将视觉中的特征衰减法移植到NLP，测试其对未知类别识别的效果。
Adapting Feature Attenuation to NLP
- 将视觉领域的COSTARR框架适配至BERT和GPT-2模型
- 在176个arXiv主题分类上，最大Softmax概率表现最佳
- 现有视觉迁移方法在高类别任务中效果有限，需针对性改进
Transformer分类器如BERT在封闭集任务中表现优异，但在面对未见类别输入时仍显脆弱，这在实际部署的NLP系统中十分常见。本文通过将计算机视觉中的特征衰减假设引入文本领域，评估其在开放集识别（OSR）中的表现，并与当前最优基线对比。具体地，我们将原本为计算机视觉设计的COSTARR框架应用于两个小型语言模型（BERT (base) 和 GPT-2），这些模型被训练用于176个arXiv主题分类任务。同时，在OOSA和AUOSCR指标下，我们评估了最大Softmax概率（MSP）、MaxLogit及温度缩放自由能分数的表现。结果表明：(i) COSTARR无需重新训练即可迁移至NLP，但相比MaxLogit或MSP无显著提升；(ii) 自由能在该高类别数设置下落后于所有其他评分方法。研究揭示了将视觉主导的OSR思想移植至语言模型的潜力与当前局限，提示需要更大规模的模型和任务定制化的衰减策略。
原文摘要 · Abstract (English)
Transformer classifiers such as BERT deliver impressive closed-set accuracy, yet they remain brittle when confronted with inputs from unseen categories--a common scenario for deployed NLP systems. We investigate Open-Set Recognition (OSR) for text by porting the feature attenuation hypothesis from computer vision to transformers and by benchmarking it against state-of-the-art baselines. Concretely, we adapt the COSTARR framework--originally designed for classification in computer vision--to two modest language models (BERT (base) and GPT-2) trained to label 176 arXiv subject areas. Alongside COSTARR, we evaluate Maximum Softmax Probability (MSP), MaxLogit, and the temperature-scaled free-energy score under the OOSA and AUOSCR metrics. Our results show (i) COSTARR extends to NLP without retraining but yields no statistically significant gain over MaxLogit or MSP, and (ii) free-energy lags behind all other scores in this high-class-count setting. The study highlights both the promise and the current limitations of transplanting vision-centric OSR ideas to language models, and points toward the need for larger backbones and task-tailored attenuation strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。