arXiv:2509.19346cs.CL2025-09被引 1

对比ChatGPT与DeepSeek用户评价,用双方法分析情感倾向。

Benchmarking ChatGPT and DeepSeek in April 2025: A Novel Dual Perspective Sentiment Analysis Using Lexicon-Based and Deep Learning Approaches

  • 结合词典法与深度学习模型,双视角分析应用评论。
  • CNN模型准确率达96.41%,优于Bi-LSTM,负向评论分类近乎完美。
  • 为大模型应用的情感分析提供新标准,适合开发者参考。

本研究提出一种新型双视角方法,分析Google Play Store上ChatGPT与DeepSeek的用户评论,融合词典法(TextBlob)与深度学习分类模型(CNN与Bi-LSTM)。采集4,000条真实用户评论,经预处理与过采样实现类别平衡,测试集含1,700条平衡样本。实验结果表明,ChatGPT获得显著更积极的情感评价。深度学习模型整体表现优于词典法,其中CNN达到96.41%准确率,负向评论分类近乎完美,中性和正面情感也保持高F1值。该研究为基于大语言模型的应用情感分析建立新方法标准,为开发者与研究人员优化以用户为中心的AI系统设计提供实践洞见。

原文摘要 · Abstract (English)

This study presents a novel dual-perspective approach to analyzing user reviews for ChatGPT and DeepSeek on the Google Play Store, integrating lexicon-based sentiment analysis (TextBlob) with deep learning classification models, including Convolutional Neural Networks (CNN) and Bidirectional Long Short Term Memory (Bi LSTM) Networks. Unlike prior research, which focuses on either lexicon-based strategies or predictive deep learning models in isolation, this study conducts an extensive investigation into user satisfaction with Large Language Model (LLM) based applications. A Dataset of 4,000 authentic user reviews was collected, which were carefully preprocessed and subjected to oversampling to achieve balanced classes. The balanced test set of 1,700 Reviews were used for model testing. Results from the experiments reveal that ChatGPT received significantly more positive sentiment than DeepSeek. Furthermore, deep learning based classification demonstrated superior performance over lexicon analysis, with CNN outperforming Bi-LSTM by achieving 96.41 percent accuracy and near perfect classification of negative reviews, alongside high F1-scores for neutral and positive sentiments. This research sets a new methodological standard for measuring sentiment in LLM-based applications and provides practical insights for developers and researchers seeking to improve user-centric AI system design.

情感分析大模型评测深度学习用户评价

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。