对比三大AI工具，GPT-4o在德语新闻评论仇恨言论检测中表现最优。
Digital Guardians: Can GPT-4, Perspective API, and Moderation API reliably detect hate speech in reader comments of German online newspapers?
- 用零样本、单样本、少样本三种提示策略测试GPT-4o
- GPT-4o比基准线高约5个百分点，综合指标领先
- 适合需要精准识别德语网络评论的媒体平台参考
近年来，网络上的有毒内容和仇恨言论日益泛滥。在线报纸和论坛的编辑需根据法律要求，仔细审查并删除读者评论，这一过程极为耗时。一些大模型服务商已提供自动化仇恨言论检测方案，包括OpenAI的GPT-4o、Google旗下Jigsaw的Perspective API以及OpenAI的Moderation API。基于专为检测在线报纸读者评论仇恨言论设计的德语测试数据集HOCON34k（含1,592个标注文本样本），本文对上述工具进行比较，并与HOCON34k基线对照。GPT-4o采用零样本、单样本和少样本三种提示方式，实验结果表明其性能优于Perspective API和Moderation API，综合马修相关系数（MCC）与F2得分，相较基线提升约5个百分点。
原文摘要 · Abstract (English)
In recent years, toxic content and hate speech have become widespread phenomena on the internet. Moderators of online newspapers and forums are now required, partly due to legal regulations, to carefully review and, if necessary, delete reader comments. This is a labor-intensive process. Some providers of large language models already offer solutions for automated hate speech detection or the identification of toxic content. These include GPT-4o from OpenAI, Jigsaw's (Google) Perspective API, and OpenAI's Moderation API. Based on the selected German test dataset HOCON34k, which was specifically created for developing tools to detect hate speech in reader comments of online newspapers, these solutions are compared with each other and against the HOCON34k baseline. The test dataset contains 1,592 annotated text samples. For GPT-4o, three different promptings are used, employing a Zero-Shot, One-Shot, and Few-Shot approach. The results of the experiments demonstrate that GPT-4o outperforms both the Perspective API and the Moderation API, and exceeds the HOCON34k baseline by approximately 5 percentage points, as measured by a combined metric of MCC and F2-score.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。