用知识蒸馏等方法提升罗马尼亚语辱骂检测效果
Enhancing Romanian Offensive Language Detection through Knowledge Distillation, Multi-Task Learning, and Data Augmentation
- 结合知识蒸馏与多任务学习优化模型
- 在罗马尼亚语数据集上准确率显著提升
- 适合语言资源少的低资源场景研究者
本文探讨了自然语言处理在人工智能中的关键作用，尤其关注对话机器人等应用。针对罗马尼亚语辱骂内容检测任务，提出三种先进方法：(1) 使用基于Transformer的神经网络进行攻击性语言识别；(2) 通过数据增强和知识蒸馏提升性能；(3) 结合多任务学习、知识蒸馏与教师退火策略，在多种数据集上实现高效训练。实验结果表明，该方法在罗马尼亚语数据集上实现了显著的性能提升，验证了其有效性。
原文摘要 · Abstract (English)
This paper highlights the significance of natural language processing (NLP) within artificial intelligence, underscoring its pivotal role in comprehending and modeling human language. Recent advancements in NLP, particularly in conversational bots, have garnered substantial attention and adoption among developers. This paper explores advanced methodologies for attaining smaller and more efficient NLP models. Specifically, we employ three key approaches: (1) training a Transformer-based neural network to detect offensive language, (2) employing data augmentation and knowledge distillation techniques to increase performance, and (3) incorporating multi-task learning with knowledge distillation and teacher annealing using diverse datasets to enhance efficiency. The culmination of these methods has yielded demonstrably improved outcomes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。