用动态注意力+对比学习提升长文本情感分类准确率
Dynamic Adaptive Attention and Supervised Contrastive Learning: A Novel Hybrid Framework for Text Sentiment Classification
- 动态调节注意力头权重，聚焦关键情感词
- 在IMDB上达94.67%准确率，比基线高1.5-2.5个百分点
- 轻量高效，适合扩展到其他文本分类任务
数字平台上的用户生成电影评论呈指数增长，准确的情感分类成为自然语言处理的核心任务。传统模型如标准BERT和循环结构常难以捕捉长文本中的远距离语义依赖，并处理模糊情感表达。本文提出一种新型混合框架，将动态自适应多头注意力与有监督对比学习无缝集成到基于BERT的Transformer编码器中。动态自适应注意力模块利用全局上下文池化向量动态调节每个注意力头的贡献，从而聚焦关键情感标记并抑制噪声。同时，有监督对比学习分支在嵌入空间中强化类内紧凑性和类间分离性。在IMDB数据集上的大量实验表明，所提模型取得94.67%的准确率，优于强基线1.5至2.5个百分点。该框架轻量、高效且易于扩展至其他文本分类任务。
原文摘要 · Abstract (English)
The exponential growth of user-generated movie reviews on digital platforms has made accurate text sentiment classification a cornerstone task in natural language processing. Traditional models, including standard BERT and recurrent architectures, frequently struggle to capture long-distance semantic dependencies and resolve ambiguous emotional expressions in lengthy review texts. This paper proposes a novel hybrid framework that seamlessly integrates dynamic adaptive multi-head attention with supervised contrastive learning into a BERT-based Transformer encoder. The dynamic adaptive attention module employs a global context pooling vector to dynamically regulate the contribution of each attention head, thereby focusing on critical sentiment-bearing tokens while suppressing noise. Simultaneously, the supervised contrastive learning branch enforces tighter intra-class compactness and larger inter-class separation in the embedding space. Extensive experiments on the IMDB dataset demonstrate that the proposed model achieves competitive performance with an accuracy of 94.67\%, outperforming strong baselines by 1.5--2.5 percentage points. The framework is lightweight, efficient, and readily extensible to other text classification tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。