分析大模型在文本网络暴力检测中的双面作用
A survey of textual cyber abuse detection using cutting-edge language models and large language models
- 用大语言模型分析社交媒体中的仇恨言论等网络暴力
- 发现大模型既能提升检测效率，也可能生成有害内容
- 适合关注AI伦理与网络治理的研究者阅读
社交媒体平台的普及催生了多种网络暴力形式，包括仇恨言论、网络欺凌、情感虐待、诱骗和性暗示。本文系统分析了这些网络暴力的表现形式，重点探讨语言模型（LMs）和大型语言模型（LLMs）如何重塑网络暴力的检测与生成机制。研究深入剖析了网络暴力传播的心理与社会动因，并揭示先进语言模型的双重角色：一方面可增强自动化检测能力，另一方面也可能被用于生成有害内容。本文旨在推动网络空间安全与伦理讨论，为理解网络暴力演化态势及技术带来的双重影响提供洞见。
原文摘要 · Abstract (English)
The success of social media platforms has facilitated the emergence of various forms of online abuse within digital communities. This abuse manifests in multiple ways, including hate speech, cyberbullying, emotional abuse, grooming, and sexting. In this paper, we present a comprehensive analysis of the different forms of abuse prevalent in social media, with a particular focus on how emerging technologies, such as Language Models (LMs) and Large Language Models (LLMs), are reshaping both the detection and generation of abusive content within these networks. We delve into the mechanisms through which social media abuse is perpetuated, exploring the psychological and social impact. Additionally, we examine the dual role of advanced language models-highlighting their potential to enhance automated detection systems for abusive behavior while also acknowledging their capacity to generate harmful content. This paper aims to contribute to the ongoing discourse on online safety and ethics, offering insights into the evolving landscape of cyberabuse and the technological innovations that both mitigate and exacerbate it.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。