构建冰岛博客评论多维度标注数据集，助力网络有害行为检测
Hotter and Colder: A New Approach to Annotating Sentiment, Emotions, and Bias in Icelandic Blog Comments
- 用GPT-4o mini自动标注80万条评论，覆盖25项任务
- 经人工校验后获1.2万条高质量标注，含1.9万次细粒度标记
- 适合研究冰岛语内容审核与网络情绪分析的学者使用
本文提出Hotter and Colder数据集，用于分析冰岛博客评论中的多种在线行为。基于前期工作，我们利用GPT-4o mini对约80万条评论进行25项任务的标注，包括情感分析、情绪识别、仇恨言论和群体刻板印象等，每条评论在5点李克特量表上自动打标。第二阶段中，对各类行为高/低概率的评论由众包工作者进行人工复核，确保数据质量。最终获得12,232条唯一标注评论和19,301次标注结果。该数据集为冰岛语内容安全研究及有害网络行为自动检测提供了关键资源。
原文摘要 · Abstract (English)
This paper presents Hotter and Colder, a dataset designed to analyze various types of online behavior in Icelandic blog comments. Building on previous work, we used GPT-4o mini to annotate approximately 800,000 comments for 25 tasks, including sentiment analysis, emotion detection, hate speech, and group generalizations. Each comment was automatically labeled on a 5-point Likert scale. In a second annotation stage, comments with high or low probabilities of containing each examined behavior were subjected to manual revision. By leveraging crowdworkers to refine these automatically labeled comments, we ensure the quality and accuracy of our dataset resulting in 12,232 uniquely annotated comments and 19,301 annotations. Hotter and Colder provides an essential resource for advancing research in content moderation and automatically detectiong harmful online behaviors in Icelandic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。