对比多种AI模型在文本与表情符号情感分析中的表现，发现新表情符号导致准确率暴跌。
Performance Evaluation of Sentiment Analysis on Text and Emoji Data Using End-to-End, Transfer Learning, Distributed and Explainable AI Models
- 用嵌入模型生成文本和表情符号向量，训练神经网络分类器
- 新表情符号测试下准确率从98%降至70%，暴露泛化瓶颈
- 分布式训练提速15%且不降精度，结合SHAP提升可解释性
在当今数字世界中，表情符号被广泛用于表达从简单到复杂的思想。因此，它们也被应用于情感分析和精准营销。本文对推文及来自Kaggle的表情符号数据集进行情感分析。由于推文为句子，我们使用通用句子编码器（USE）和句子双向编码器表示（SBERT）生成嵌入，并用于训练标准全连接神经网络（NN）和LSTM NN模型。结果显示，两种模型在文本分类上的准确率均约为98%。然而，当验证集包含训练集中未出现的表情符号时，两类模型的准确率急剧下降至70%。此外，采用分布式训练方法替代传统单线程模型以提升可扩展性，使运行时间减少约15%且不影响准确率。最后，通过SHAP算法实现可解释AI，分析模型行为并检测特征集中的潜在偏差。
原文摘要 · Abstract (English)
Emojis are being frequently used in todays digital world to express from simple to complex thoughts more than ever before. Hence, they are also being used in sentiment analysis and targeted marketing campaigns. In this work, we performed sentiment analysis of Tweets as well as on emoji dataset from the Kaggle. Since tweets are sentences we have used Universal Sentence Encoder (USE) and Sentence Bidirectional Encoder Representations from Transformers (SBERT) end-to-end sentence embedding models to generate the embeddings which are used to train the Standard fully connected Neural Networks (NN), and LSTM NN models. We observe the text classification accuracy was almost the same for both the models around 98 percent. On the contrary, when the validation set was built using emojis that were not present in the training set then the accuracy of both the models reduced drastically to 70 percent. In addition, the models were also trained using the distributed training approach instead of a traditional singlethreaded model for better scalability. Using the distributed training approach, we were able to reduce the run-time by roughly 15% without compromising on accuracy. Finally, as part of explainable AI the Shap algorithm was used to explain the model behaviour and check for model biases for the given feature set.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。