arXiv:2412.16417cs.LGcs.CL2024-12被引 2

用大模型识别社交媒体欺凌角色,准确率达89.3%。

Identifying Cyberbullying Roles in Social Media

  • 基于RoBERTa等大模型,结合数据增强提升角色识别性能。
  • 最佳模型F1达83.5%,阈值优化后提升至89.3%。
  • 适合研究网络欺凌、心理健康与内容安全的学者与工程师。

社交媒体重塑了全球沟通方式,但同时也加剧了网络欺凌问题,严重威胁儿童与青少年的心理健康。准确识别欺凌事件中个体的角色对大规模应对该问题至关重要。本研究探索使用机器学习模型检测社交互动中的欺凌角色。在分析AMiCA数据集并解决类别不平衡问题后,评估了四种底层大语言模型(BERT、RoBERTa、T5、GPT-2)的表现。实验表明,过采样技术可有效提升模型性能。最佳模型为微调后的RoBERTa,在过采样数据上达到83.5%的整体F1分数,经预测阈值调整后提升至89.3%,未阈值化时的前两分类F1为95.7%。该方法优于已有模型。进一步分析显示,模型在样本多、语境清晰的类别(如旁观者其他)表现良好,但在样本少、语义模糊的类别(如协助型旁观者、施暴者与受害者)上仍存在困难。研究揭示了当前在有限数据与复杂场景下构建高精度模型的优劣与挑战。

原文摘要 · Abstract (English)

Social media has revolutionized communication, allowing people worldwide to connect and interact instantly. However, it has also led to increases in cyberbullying, which poses a significant threat to children and adolescents globally, affecting their mental health and well-being. It is critical to accurately detect the roles of individuals involved in cyberbullying incidents to effectively address the issue on a large scale. This study explores the use of machine learning models to detect the roles involved in cyberbullying interactions. After examining the AMiCA dataset and addressing class imbalance issues, we evaluate the performance of various models built with four underlying LLMs (i.e., BERT, RoBERTa, T5, and GPT-2) for role detection. Our analysis shows that oversampling techniques help improve model performance. The best model, a fine-tuned RoBERTa using oversampled data, achieved an overall F1 score of 83.5%, increasing to 89.3% after applying a prediction threshold. The top-2 F1 score without thresholding was 95.7%. Our method outperforms previously proposed models. After investigating the per-class model performance and confidence scores, we show that the models perform well in classes with more samples and less contextual confusion (e.g., Bystander Other), but struggle with classes with fewer samples (e.g., Bystander Assistant) and more contextual ambiguity (e.g., Harasser and Victim). This work highlights current strengths and limitations in the development of accurate models with limited data and complex scenarios.

网络欺凌角色识别大模型应用文本分类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。