用生成模型增强数据,让仇恨言论检测更公平地覆盖弱势群体。
A Target-Aware Analysis of Data Augmentation for Hate Speech Detection
- 结合传统增强与生成模型,合成3万条新数据提升检测效果。
- 对种族、宗教、残疾等类别,F1值提升超10%。
- 适合关注公平性与包容性的仇恨言论研究者。
仇恨言论是社交媒体广泛使用带来的主要威胁之一,尽管已有诸多应对措施,但针对残障主义、年龄歧视等少数群体的标注数据匮乏,导致现有检测系统在弱势身份群体上表现不佳。鉴于大语言模型生成高质量文本的潜力,本文探索利用生成模型扩充Measuring Hate Speech语料库中的1,000条原始帖子,生成约3万条合成样本,涵盖简单增强方法及自回归与序列到序列两类生成模型。实验表明,传统数据增强方法通常优于纯生成模型,但二者结合可实现最佳性能。例如,在起源、宗教和残疾等类别上,使用增强数据训练的分类器相比无增强基线,F1值提升超过10%。本工作推动了更高效、更公平且更具包容性的仇恨言论检测系统的发展。
原文摘要 · Abstract (English)
Hate speech is one of the main threats posed by the widespread use of social networks, despite efforts to limit it. Although attention has been devoted to this issue, the lack of datasets and case studies centered around scarcely represented phenomena, such as ableism or ageism, can lead to hate speech detection systems that do not perform well on underrepresented identity groups. Given the unpreceded capabilities of LLMs in producing high-quality data, we investigate the possibility of augmenting existing data with generative language models, reducing target imbalance. We experiment with augmenting 1,000 posts from the Measuring Hate Speech corpus, an English dataset annotated with target identity information, adding around 30,000 synthetic examples using both simple data augmentation methods and different types of generative models, comparing autoregressive and sequence-to-sequence approaches. We find traditional DA methods to often be preferable to generative models, but the combination of the two tends to lead to the best results. Indeed, for some hate categories such as origin, religion, and disability, hate speech classification using augmented data for training improves by more than 10% F1 over the no augmentation baseline. This work contributes to the development of systems for hate speech detection that are not only better performing but also fairer and more inclusive towards targets that have been neglected so far.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。