对比12种假新闻检测方法,发现模型在跨领域时效果大幅下降。
An Experimental Comparison of the Most Popular Approaches to Fake News Detection
- 系统评估12种主流假新闻检测方法,涵盖传统与深度学习模型。
- 同域测试准确率超85%,跨域测试性能骤降30%以上。
- 大语言模型在零样本场景表现突出,适合缺乏标注数据的场景。
近年来,假新闻检测在公共讨论和科研中备受关注。尽管检测技术不断进步,但虚假信息的生成与传播因大语言模型(LLMs)和社交媒体的放大效应而愈发复杂。本文对12种代表性假新闻检测方法进行了批判性评估,涵盖传统机器学习、深度学习、Transformer及专用跨领域架构。在10个公开数据集上进行实验,这些数据集在体裁、来源、主题和标注逻辑上各不相同。将文本仅限英文的假新闻检测任务统一为二分类(真实/虚假),以确保评估一致性。我们承认不同数据集标签语义存在差异,统一分类会损失部分语义细节。每个数据集视为独立领域,开展同域、多域与跨域实验,模拟现实中的领域偏移与分布外数据场景。微调模型在同域表现优异,但泛化能力差;跨域架构虽可缓解差距,但需大量数据;而大语言模型通过零样本和少样本学习展现出良好前景。鉴于数据集固有的混淆因素及预训练暴露可能,结果应理解为在此英文纯文本协议下的鲁棒性评估。
原文摘要 · Abstract (English)
In recent years, fake news detection has received increasing attention in public debate and scientific research. Despite advances in detection techniques, the production and spread of false information have become more sophisticated, driven by Large Language Models (LLMs) and the amplification power of social media. We present a critical assessment of 12 representative fake news detection approaches, spanning traditional machine learning, deep learning, transformers, and specialized cross-domain architectures. We evaluate these methods on 10 publicly available datasets differing in genre, source, topic, and labeling rationale. We address text-only English fake news detection as a binary classification task by harmonizing labels into "Real" and "Fake" to ensure a consistent evaluation protocol. We acknowledge that label semantics vary across datasets and that harmonization inevitably removes such semantic nuances. Each dataset is treated as a distinct domain. We conduct in-domain, multi-domain and cross-domain experiments to simulate real-world scenarios involving domain shift and out-of-distribution data. Fine-tuned models perform well in-domain but struggle to generalize. Cross-domain architectures can reduce this gap but are data-hungry, while LLMs offer a promising alternative through zero- and few-shot learning. Given inherent dataset confounds and possible pre-training exposure, results should be interpreted as robustness evaluations within this English, text-only protocol.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。