用双向LSTM微调BERT,提升电影评论情感分析精度。
Fine-tuning BERT with Bidirectional LSTM for Fine-grained Movie Reviews Sentiment Analysis
- BERT+双向LSTM融合结构,兼顾上下文与序列特征
- 二分类准确率达97.67%,五分类达59.48%,优于基线模型
- 适合需要细粒度情感判断的影视评论分析场景
情感分析在理解公众观点、社交媒体监控、品牌识别和客户满意度评估中至关重要,已成为自然语言处理的重要研究方向。现有方法多聚焦于二分类或细粒度分类,但常忽略二者兼顾。本文提出将预训练BERT与双向LSTM结合,用于电影评论的二分类与五分类情感分析。通过逐条分类后计算整体情感极性,我们在IMDb与SST-5等基准数据集上验证效果。引入SMOTE与NLPAUG增强数据以提升泛化能力,并设计启发式算法从输出向量计算总体极性。结果表明:二分类准确率达97.67%,超越NB-weighted-BON+dv-cosine 0.27%;五分类在SST-5上达59.48%,较BERT-large提升3.6%,优于RoBERTa+large+Self-explaining(55.5%),性能接近当前最优水平。
原文摘要 · Abstract (English)
Sentiment Analysis (SA) is instrumental in understanding peoples viewpoints facilitating social media monitoring recognizing products and brands and gauging customer satisfaction. Consequently SA has evolved into an active research domain within Natural Language Processing (NLP). Many approaches outlined in the literature devise intricate frameworks aimed at achieving high accuracy, focusing exclusively on either binary sentiment classification or fine-grained sentiment classification. In this paper our objective is to fine-tune the pre-trained BERT model with Bidirectional LSTM (BiLSTM) to enhance both binary and fine-grained SA specifically for movie reviews. Our approach involves conducting sentiment classification for each review followed by computing the overall sentiment polarity across all reviews. We present our findings on binary classification as well as fine-grained classification utilizing benchmark datasets. Additionally we implement and assess two accuracy improvement techniques Synthetic Minority Oversampling Technique (SMOTE) and NLP Augmenter (NLPAUG) to bolster the models generalization in fine-grained sentiment classification. Finally a heuristic algorithm is employed to calculate the overall polarity of predicted reviews from the BERT+BiLSTM output vector. Our approach performs comparably with state-of-the-art (SOTA) techniques in both classifications. For instance in binary classification we achieve 97.67% accuracy surpassing the leading SOTA model NB-weighted-BON+dv-cosine by 0.27% on the renowned IMDb dataset. Conversely for five-class classification on SST-5 while the top SOTA model RoBERTa+large+Self-explaining attains 55.5% accuracy our model achieves 59.48% accuracy surpassing the BERT-large baseline by 3.6%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。