用机器学习区分真假新闻质量,准确率超87%
Classification of worldwide news articles by perceived quality, 2018-2024
- 基于140万篇新闻构建数据集,用语言特征分类高低质量
- ModernBERT-large模型准确率达87.44%,优于传统机器学习
- 适合关注虚假信息检测与媒体质量评估的研究者
本研究探讨了监督式机器学习与深度学习模型能否有效区分感知低质量与高质量新闻文章。使用2018-2024年来自Common Crawl的1,412,272篇英文新闻文章,构建新数据集,结合专家对579个网站的质量评级中位数划分,形成约各706,000篇的高低质量类别,每篇文章包含194个网站级语言特征。传统机器学习分类器如随机森林表现良好(准确率0.7355,ROC AUC 0.8131)。深度学习方面,ModernBERT-large(256上下文长度)表现最佳(准确率0.8744,ROC-AUC 0.9593,F1 0.8739),DistilBERT-base(512上下文长度)次之(准确率0.8685,ROC-AUC 0.9554),DistilBERT-base(256上下文长度)达0.8478准确率与0.9407 ROC-AUC,ModernBERT-base(256上下文长度)获0.8569准确率与0.9470 ROC-AUC。结果表明,可通过传统与深度学习模型有效区分全球新闻的感知质量。
原文摘要 · Abstract (English)
This study explored whether supervised machine learning and deep learning models can effectively distinguish perceived lower-quality news articles from perceived higher-quality news articles. 3 machine learning classifiers and 3 deep learning models were assessed using a newly created dataset of 1,412,272 English news articles from the Common Crawl over 2018-2024. Expert consensus ratings on 579 source websites were split at the median, creating perceived low and high-quality classes of about 706,000 articles each, with 194 linguistic features per website-level labelled article. Traditional machine learning classifiers such as the Random Forest demonstrated capable performance (0.7355 accuracy, 0.8131 ROC AUC). For deep learning, ModernBERT-large (256 context length) achieved the best performance (0.8744 accuracy; 0.9593 ROC-AUC; 0.8739 F1), followed by DistilBERT-base (512 context length) at 0.8685 accuracy and 0.9554 ROC-AUC. DistilBERT-base (256 context length) reached 0.8478 accuracy and 0.9407 ROC-AUC, while ModernBERT-base (256 context length) attained 0.8569 accuracy and 0.9470 ROC-AUC. These results suggest that the perceived quality of worldwide news articles can be effectively differentiated by traditional CPU-based machine learning classifiers and deep learning classifiers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。