用DeBERTa-v3-large实现日语文本情绪二分类,准确率达86%
Performance Evaluation of Emotion Classification in Japanese Using RoBERTa and DeBERTa
- 微调DeBERTa-v3-large模型处理八类情绪的二值判断
- 平均准确率86.0%,低频情绪也保持稳定性能
- 适合日语情绪分析研究者与工业应用开发者
实际应用如社交媒体监控和客户反馈分析需要高精度的日语文本情绪检测,但资源匮乏和类别不平衡限制了模型表现。本研究旨在构建一个可预测八种Plutchik情绪是否存在、准确率高的模型。基于WRIME语料库,将读者平均强度评分转化为二值标签,微调四种预训练语言模型(BERT、RoBERTa、DeBERTa-v3-base、DeBERTa-v3-large),并对比两个大语言模型(TinySwallow-1.5B-Instruct和ChatGPT-4o)。以准确率和F1分数为评估指标。结果显示,DeBERTa-v3-large在平均准确率(0.860)和平均F1(0.662)上均优于其他模型,对高频(如喜悦、期待)和低频(如愤怒、信任)情绪均有良好表现;而大语言模型表现较差,ChatGPT-4o和TinySwallow-1.5B-Instruct的平均F1分别为0.527和0.292。结论:微调后的DeBERTa-v3-large是当前最可靠的日语文本情绪二分类方案。模型已发布为pip包(pip install deberta-emotion-predictor)。未来工作应增强稀有情绪数据、压缩模型规模,并探索提示工程提升大模型性能。本文正在新世代计算期刊审稿中。
原文摘要 · Abstract (English)
Background Practical applications such as social media monitoring and customer-feedback analysis require accurate emotion detection for Japanese text, yet resource scarcity and class imbalance hinder model performance. Objective This study aims to build a high-accuracy model for predicting the presence or absence of eight Plutchik emotions in Japanese sentences. Methods Using the WRIME corpus, we transform reader-averaged intensity scores into binary labels and fine-tune four pre-trained language models (BERT, RoBERTa, DeBERTa-v3-base, DeBERTa-v3-large). For context, we also assess two large language models (TinySwallow-1.5B-Instruct and ChatGPT-4o). Accuracy and F1-score serve as evaluation metrics. Results DeBERTa-v3-large attains the best mean accuracy (0.860) and F1-score (0.662), outperforming all other models. It maintains robust F1 across both high-frequency emotions (e.g., Joy, Anticipation) and low-frequency emotions (e.g., Anger, Trust). The LLMs lag, with ChatGPT-4o and TinySwallow-1.5B-Instruct scoring 0.527 and 0.292 in mean F1, respectively. Conclusion The fine-tuned DeBERTa-v3-large model currently offers the most reliable solution for binary emotion classification in Japanese. We release this model as a pip-installable package (pip install deberta-emotion-predictor). Future work should augment data for rare emotions, reduce model size, and explore prompt engineering to improve LLM performance. This manuscript is under review for possible publication in New Generation Computing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。