用表情符号增强的混合模型,提升Twitch直播聊天毒性检测准确率。
ToxiTwitch: Toward Emote-Aware Hybrid Moderation for Live Streaming Platforms
- 结合文本与表情符号的嵌入表示,融合大模型与传统分类器
- 在特定频道训练下准确率达80%,比BERT提升13个百分点
- 适合关注直播平台内容安全、需处理多模态聊天的开发者
Twitch等直播平台的快速发展带来了复杂的毒性行为监管挑战。传统方法如人工标注和关键词过滤虽有效,但面对高并发、快节奏且富含上下文的聊天环境,人工审核难以规模化,且自身易受骚扰。大语言模型(如DeepSeek-R1-Distill、Llama-3-8B-Instruct)为理解包含表情符号的复杂多模态交流提供了新可能。本文开展针对Twitch的毒性检测方法探索性比较,发现融入表情符号可显著提升检测效果。为此提出ToxiTwitch:一种将LLM生成的文本与表情符号嵌入结合传统机器学习分类器(随机森林、SVM)的混合模型。案例研究显示,该方法在频道特定训练下准确率最高达80%,相较BERT提升13个百分点,F1分数达76%。本工作为表情符号感知的毒性检测在Twitch上的应用提供初步探索,揭示其潜在挑战与局限。
原文摘要 · Abstract (English)
The rapid growth of live-streaming platforms such as Twitch has introduced complex challenges in moderating toxic behavior. Traditional moderation approaches, such as human annotation and keyword-based filtering, have demonstrated utility, but human moderators on Twitch constantly struggle to scale effectively in the fast-paced, high-volume, and context-rich chat environment of the platform while also facing harassment themselves. Recent advances in large language models (LLMs), such as DeepSeek-R1-Distill and Llama-3-8B-Instruct, offer new opportunities for toxicity detection, especially in understanding nuanced, multimodal communication involving emotes. In this work, we present an exploratory comparison of toxicity detection approaches tailored to Twitch. Our analysis reveals that incorporating emotes improves the detection of toxic behavior. To this end, we introduce ToxiTwitch, a hybrid model that combines LLM-generated embeddings of text and emotes with traditional machine learning classifiers, including Random Forest and SVM. In our case study, the proposed hybrid approach reaches up to 80 percent accuracy under channel-specific training (with 13 percent improvement over BERT and F1-score of 76 percent). This work is an exploratory study intended to surface challenges and limits of emote-aware toxicity detection on Twitch.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。