arXiv:2507.14022cs.CLcs.LG2025-07

用认知比较法选最优情感分析模型,兼顾精度与效率。

CPC-CMS: Cognitive Pairwise Comparison Classification Model Selection Framework for Document-level Sentiment Analysis

  • 基于专家判断赋权,综合多指标筛选最佳模型
  • ALBERT在三数据集上综合表现最优,时间因素下无绝对优胜者
  • 方法稳定可靠,适用于各类分类任务的模型选择

本文提出认知配对比较分类模型选择(CPC-CMS)框架,用于文档级情感分析。基于专家知识判断,计算准确率、精确率、召回率、F1值、特异性、马修斯相关系数(MCC)、科恩κ系数(Kappa)和效率等评估指标的权重。选用朴素贝叶斯(NB)、线性支持向量分类(LSVC)、随机森林、逻辑回归、极端梯度提升(XGBoost)、长短期记忆网络(LSTM)和轻量双向编码器表示(ALBERT)作为基准分类模型。构建加权决策矩阵,依据指标权重选择最优模型。在三个公开社交媒体数据集上验证了该框架的可行性。模拟结果显示,在不考虑时间因素时,ALBERT在所有数据集上表现最佳;若包含时间因素,则无单一模型持续领先。与其他聚合排序方法(AHP、TOPSIS、MOORA)对比,结论一致,尽管具体数值和排名略有差异。通过斯皮尔曼等级相关检验,验证了框架的稳健性。该框架可推广至其他领域分类应用。

原文摘要 · Abstract (English)

This study proposes the Cognitive Pairwise Comparison Classification Model Selection (CPC-CMS) framework for document-level sentiment analysis. The CPC, based on expert knowledge judgment, is used to calculate the weights of evaluation criteria, including accuracy, precision, recall, F1-score, specificity, Matthews Correlation Coefficient (MCC), Cohen's Kappa (Kappa), and efficiency. Naive Bayes (NB), Linear Support Vector Classification (LSVC), Random Forest, Logistic Regression, Extreme Gradient Boosting (XGBoost), Long Short-Term Memory (LSTM), and A Lite Bidirectional Encoder Representations from Transformers (ALBERT) are chosen as classification baseline models. A weighted decision matrix consisting of classification evaluation scores with respect to criteria weights is formed to select the best classification model for a classification problem. Three open social media datasets are used to demonstrate the feasibility of the proposed CPC-CMS. Based on our simulation, for evaluation results excluding the time factor, ALBERT performs best across all three datasets; if the time factor is included, no single model consistently outperforms the others. Through comparison, these conclusions are also supported by other aggregation and ranking methods, including Analytic Hierarchy Process (AHP), Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) and Multi-Objective Optimization by Ratio Analysis (MOORA), although aggregation values and ranks may vary. A sensitivity analysis using Spearman's Rank Correlation Test demonstrates the robustness of the proposed CPC-CMS framework. The CPC-CMS can be applied to other classification applications in various domains.

情感分析模型选择多指标决策ALBERT

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。