用关键词筛选+词汇扩充，让BERT更高效识别网络辱骂内容。
Feature Selection Empowered BERT for Detection of Hate Speech with Vocabulary Augmentation
- 选75%最有信息量样本训练，减少数据量
- 在原词典中加入网络黑话变体，提升识别率
- 适合需要快速部署的平台内容审核场景
社交媒体上的辱骂性言论持续演变，常使用新出现的俚语和变形词汇规避检测。本文提出一种数据高效的BERT微调策略，在不降低性能的前提下显著减少训练集规模。通过基于TF-IDF的样本选择机制，仅保留最具信息量的75%样本以降低训练开销。为弥补BERT原生词典对新兴辱骂术语覆盖不足的问题，我们引入领域内常见俚语及词汇变体对分词器进行增强。在常用仇恨言论数据集上的实验表明，该方法在保持竞争力表现的同时提升了计算效率，展现出在可扩展、自适应的内容安全治理中的潜力。
原文摘要 · Abstract (English)
Abusive speech on social media poses a persistent and evolving challenge, driven by the continuous emergence of novel slang and obfuscated terms designed to circumvent detection systems. In this work, we present a data efficient strategy for fine tuning BERT on hate speech classification by significantly reducing training set size without compromising performance. Our approach employs a TF IDF-based sample selection mechanism to retain only the most informative 75 percent of examples, thereby minimizing training overhead. To address the limitations of BERT's native vocabulary in capturing evolving hate speech terminology, we augment the tokenizer with domain-specific slang and lexical variants commonly found in abusive contexts. Experimental results on a widely used hate speech dataset demonstrate that our method achieves competitive performance while improving computational efficiency, highlighting its potential for scalable and adaptive abusive content moderation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。