用大模型预测舆情热度,低热度事件效果好。
Research on Predicting Public Opinion Event Heat Levels Based on Large Language Models
- 基于聚类划分四档舆情热度,构建1000条评估数据集。
- GPT-4o和DeepseekV2在有参考案例时准确率达41.5%。
- 低热度事件预测准确率超70%,适合舆情监测初探者。
近年来,随着大语言模型的快速发展,GPT-4o等模型在多项语言任务中表现超越人类。本研究提出一种基于大语言模型的舆情事件热度预测方法。首先,对2022年7月至2023年12月间收集的62,836条中文热点事件数据进行预处理与分类;随后,基于事件的线上传播热度指数,使用MiniBatchKMeans算法自动聚类,将事件划分为四个热度等级(从低到极高)。从中每级随机选取250条事件,共1,000条构建评估数据集。在评估过程中,采用多种大语言模型,在无参考案例和有相似案例参考两种场景下测试其热度预测能力。结果显示,GPT-4o与DeepseekV2在有参考案例时表现最佳,准确率分别为41.4%和41.5%。尽管整体准确率较低,但低热度(第1级)事件的预测准确率分别达到73.6%和70.4%。此外,准确率随热度等级升高呈下降趋势,这与实际数据在各等级间分布不均有关。表明未来若具备更均衡的数据集,基于大语言模型的舆情热度预测具有重要研究潜力。
原文摘要 · Abstract (English)
In recent years, with the rapid development of large language models, serval models such as GPT-4o have demonstrated extraordinary capabilities, surpassing human performance in various language tasks. As a result, many researchers have begun exploring their potential applications in the field of public opinion analysis. This study proposes a novel large-language-models-based method for public opinion event heat level prediction. First, we preprocessed and classified 62,836 Chinese hot event data collected between July 2022 and December 2023. Then, based on each event's online dissemination heat index, we used the MiniBatchKMeans algorithm to automatically cluster the events and categorize them into four heat levels (ranging from low heat to very high heat). Next, we randomly selected 250 events from each heat level, totalling 1,000 events, to build the evaluation dataset. During the evaluation process, we employed various large language models to assess their accuracy in predicting event heat levels in two scenarios: without reference cases and with similar case references. The results showed that GPT-4o and DeepseekV2 performed the best in the latter case, achieving prediction accuracies of 41.4% and 41.5%, respectively. Although the overall prediction accuracy remains relatively low, it is worth noting that for low-heat (Level 1) events, the prediction accuracies of these two models reached 73.6% and 70.4%, respectively. Additionally, the prediction accuracy showed a downward trend from Level 1 to Level 4, which correlates with the uneven distribution of data across the heat levels in the actual dataset. This suggests that with the more robust dataset, public opinion event heat level prediction based on large language models will have significant research potential for the future.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。