用合成数据微调大模型，提升游戏聊天多类毒性检测效果
PSK@EEUCA 2026: Fine-Tuning Large Language Models with Synthetic Data Augmentation for Multi-Class Toxicity Detection in Gaming Chat
- 基于Llama 3.1 8B，结合5%合成数据微调，提升分类性能
- 在测试集上取得0.6234的F1-macro分数，排名第四
- 揭示验证集表现高但测试迁移差的'验证陷阱'现象
本文介绍我们在EEUCA 2026共享任务中针对游戏社区毒性行为识别的系统。任务要求将《坦克世界》聊天消息分为六类：非毒性、辱骂/激怒、其他冒犯性内容、仇恨/骚扰、威胁和极端主义。我们探索了编码器模型、指令微调的大语言模型（采用LoRA）、层次分类、一对一分类及多种集成方法。最优系统结合了Llama 3.1 8B与经过精心校准的5%合成数据增强，在测试集上取得0.6234的F1-macro得分，位列35支参赛队伍中的第4名。我们对数据集的标注模式及其对模型泛化的影响进行了深入分析，发现一种关键的“验证陷阱”现象：验证性能越高，测试集迁移能力越差。
原文摘要 · Abstract (English)
This paper describes our system for the EEUCA 2026 Shared Task on Understanding Toxic Behavior in Gaming Communities. The task involves classifying World of Tanks chat messages into six toxicity categories: Non-toxic, Insults/Flaming, Other Offensive, Hate/Harassment, Threats, and Extremism. We explore multiple approaches including encoder-based models, instruction-tuned LLMs with LoRA fine-tuning, hierarchical classification, one-vs-rest strategies, and various ensemble methods. Our best system combines Llama 3.1 8B with carefully calibrated 5\% synthetic data augmentation, achieving an F1-macro score of 0.6234 on the test set, placing 4th out of 35 participating teams. We provide extensive analysis of the dataset's annotation patterns and their impact on model generalization, revealing a critical ''validation trap'' phenomenon where high validation performance correlates with poor test transfer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。