用注意力池化提升文本异常检测准确率
AP-OOD: Attention Pooling for Out-of-Distribution Detection
- 基于注意力机制聚合词元特征，替代简单平均
- 无监督下FPR95降低至4.67%（原27.84%）
- 适合标注数据少的文本异常检测场景
分布外（OOD）检测将高维数据映射为标量得分，对模型可靠部署至关重要。当前研究的核心挑战是如何有效利用和聚合语言模型中的词元嵌入以获得OOD得分。本文提出AP-OOD，一种新颖的文本OOD检测方法，超越传统平均聚合，充分利用词元级信息。该方法为半监督框架，可灵活在无监督与有监督之间插值，支持有限的辅助异常数据使用。实验表明，AP-OOD在文本OOD检测上达到新基准：在无监督设置下，XSUM摘要任务中FPR95从27.84%降至4.67%，WMT15 En-Fr翻译任务中从77.08%降至70.37%。
原文摘要 · Abstract (English)
Out-of-distribution (OOD) detection, which maps high-dimensional data into a scalar OOD score, is critical for the reliable deployment of machine learning models. A key challenge in recent research is how to effectively leverage and aggregate token embeddings from language models to obtain the OOD score. In this work, we propose AP-OOD, a novel OOD detection method for natural language that goes beyond simple average-based aggregation by exploiting token-level information. AP-OOD is a semi-supervised approach that flexibly interpolates between unsupervised and supervised settings, enabling the use of limited auxiliary outlier data. Empirically, AP-OOD sets a new state of the art in OOD detection for text: in the unsupervised setting, it reduces the FPR95 (false positive rate at 95% true positives) from 27.84% to 4.67% on XSUM summarization, and from 77.08% to 70.37% on WMT15 En-Fr translation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。