arXiv:2605.01727cs.AIcs.CY2026-05中稿 · the 2nd Workshop o…

大模型更易把娱乐新闻误判为假新闻,不同模型差异明显。

Are LLMs More Skeptical of Entertainment News?

  • 用同一数据集测试四种前沿模型对娱乐与硬新闻的误判率差异。
  • DeepSeek-V3.2和GPT-5.2在娱乐新闻上误判率高出10.1%和8.8%。
  • 模型对娱乐新闻存在认知偏见,需按新闻类型分层评估可信度。

大型语言模型(LLMs)被广泛用于自动化新闻可信度评估,但其在不同新闻体裁间是否保持一致标准尚不明确。本文基于FakeNewsNet中的GossipCop数据集,采用同数据集设计,考察零样本LLM是否更倾向于将真实娱乐新闻误判为虚假新闻。结果显示,四种前沿模型中,DeepSeek-V3.2与GPT-5.2分别表现出10.1%与8.8%的假阳性率差距(均p < .001),而Claude Opus 4.6与Gemini 3 Flash则无显著差异。风格替换实验仅带来有限且不一致的变化,表明该偏差并非单纯由文体特征引起。提示词干预虽可缓解部分模型的误判,但不具备通用性:将模型设定为娱乐新闻核查者可使DeepSeek-V3.2的假阳性降低约50%,且未造成召回率下降,但对GPT-5.2帮助甚微。探索性定性编码揭示两类常见错误模式:将私人生活陈述视为不可验证,以及贬低娱乐新闻的求知论地位。结果表明,整体准确率可能掩盖了合法新闻内部的系统性误判。我们主张,基于大模型的可信度评估不仅应判断真假,还需识别新闻体裁的合法性差异,评估应包含按体裁分层的假阳性分析。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly used for automated news credibility assessment, yet it remains unclear whether they apply even-handed standards across journalistic genres. We examine whether zero-shot LLMs are more likely to misclassify legitimate entertainment news as fake than legitimate hard news, using a within-dataset design on GossipCop from FakeNewsNet. Across four frontier models, we find a clear but model-specific genre asymmetry: DeepSeek-V3.2 and GPT-5.2 show false-positive-rate gaps of 10.1 and 8.8 percentage points, respectively (both $p < .001$), whereas Claude Opus 4.6 and Gemini 3 Flash show no comparable difference. A style-swap experiment yields only limited and inconsistent changes, suggesting that the asymmetry is not reducible to stylistic register alone. Prompt-based mitigation is likewise possible but not generic: framing the model as an entertainment-news fact-checker reduces false positives for DeepSeek-V3.2 by about 50\% without detectable recall loss, but offers little improvement for GPT-5.2. Exploratory qualitative coding further suggests two recurring error patterns in sampled false positives: treating private-life claims as inherently unverifiable and discounting entertainment journalism as an epistemically weaker genre. Taken together, these findings show that aggregate performance metrics can obscure structured false positives within legitimate journalism. We argue that LLM-based credibility assessment may not only evaluate truth claims but also differentially recognize the legitimacy of journalistic genres, and that evaluation should therefore include genre-stratified false-positive analysis alongside overall accuracy.

大模型新闻可信度偏见检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。