用AI标注+人工校验数据，对比BERT与大模型的假新闻检测效果
Fake News Detection: Comparative Evaluation of BERT-like Models and Large Language Models with Generative AI-Annotated Data
- 用GPT-4辅助标注并人工验证构建新数据集
- BERT类模型分类性能更优，大模型抗干扰更强
- AI标注加人工审核比弱标签提升检测效果
假新闻对现代公共舆论和社会稳定构成重大威胁。本研究对比了基于BERT的编码器模型与自回归解码器模型（如大语言模型）在假新闻检测中的表现。我们构建了一个由GPT-4辅助标注、经人工专家验证的数据集，确保标签可靠性。两类模型均在此数据集上微调，并提出一种指令微调的LLM结合多数投票的推理策略生成标签。分析显示，编码器模型在分类任务中普遍优于解码器模型，而解码器模型在文本扰动下表现出更强鲁棒性。相较于弱标签（远程监督）数据，采用人工智能标注加人工审核的数据显著提升分类性能。本研究验证了结合AI标注与人工监督的有效性，并展示了不同模型家族在假新闻检测中的表现差异。
原文摘要 · Abstract (English)
Fake news poses a significant threat to public opinion and social stability in modern society. This study presents a comparative evaluation of BERT-like encoder-only models and autoregressive decoder-only large language models (LLMs) for fake news detection. We introduce a dataset of news articles labeled with GPT-4 assistance (an AI-labeling method) and verified by human experts to ensure reliability. Both BERT-like encoder-only models and LLMs were fine-tuned on this dataset. Additionally, we developed an instruction-tuned LLM approach with majority voting during inference for label generation. Our analysis reveals that BERT-like models generally outperform LLMs in classification tasks, while LLMs demonstrate superior robustness against text perturbations. Compared to weak labels (distant supervision) data, the results show that AI labels with human supervision achieve better classification results. This study highlights the effectiveness of combining AI-based annotation with human oversight and demonstrates the performance of different families of machine learning models for fake news detection
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。