首个全面的NLP异常检测基准，助力识别恶意内容与诈骗信息。
NLP-ADBench: NLP Anomaly Detection Benchmark
- 构建涵盖8个数据集的统一评估框架，整合19种主流算法。
- 基于OpenAI嵌入的两阶段方法表现最优，超越专用端到端模型。
- 适合安全、内容审核与行为分析领域的研究人员使用。
异常检测（AD）是机器学习中的关键任务，广泛应用于欺诈识别、内容审核和用户行为分析。然而，自然语言处理（NLP）场景下的异常检测研究仍相对不足，制约了对有害内容、钓鱼攻击和垃圾评论的有效识别。本文提出NLP-ADBench，目前最全面的NLP异常检测基准，包含8个精心筛选的数据集和19种前沿算法。这些方法涵盖3种端到端模型与16种两阶段策略，后者将经典非异常检测方法适配至BERT和OpenAI的语言嵌入。实验表明，无单一模型在所有数据集上占据绝对优势，凸显自动化模型选择的必要性；且基于Transformer的嵌入在两阶段方法中持续领先，其中OpenAI嵌入性能优于BERT。项目代码已开源（https://github.com/USC-FORTIS/NLP-ADBench），为未来研究提供统一平台。
原文摘要 · Abstract (English)
Anomaly detection (AD) is an important machine learning task with applications in fraud detection, content moderation, and user behavior analysis. However, AD is relatively understudied in a natural language processing (NLP) context, limiting its effectiveness in detecting harmful content, phishing attempts, and spam reviews. We introduce NLP-ADBench, the most comprehensive NLP anomaly detection (NLP-AD) benchmark to date, which includes eight curated datasets and 19 state-of-the-art algorithms. These span 3 end-to-end methods and 16 two-step approaches that adapt classical, non-AD methods to language embeddings from BERT and OpenAI. Our empirical results show that no single model dominates across all datasets, indicating a need for automated model selection. Moreover, two-step methods with transformer-based embeddings consistently outperform specialized end-to-end approaches, with OpenAI embeddings outperforming those of BERT. We release NLP-ADBench at https://github.com/USC-FORTIS/NLP-ADBench, providing a unified framework for NLP-AD and supporting future investigations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。