构建多策略生成假新闻数据集，揭示现有检测模型在混合真伪内容前的脆弱性
Many Ways to Be Fake: Benchmarking Fake News Detection Under Strategy-Driven AI Generation
- 设计多种策略驱动的提示生成管道，合成6798篇高仿真假新闻
- 先进模型对纯虚构内容检测接近饱和，但对嵌套真实信息的隐性谎言失效
- 适合关注人机协作造假风险与检测鲁棒性的研究者
大型语言模型的进展使得大规模生成高度流畅且具有欺骗性的新闻类内容成为可能。以往研究常将假新闻检测视为二分类任务，但现代假新闻越来越多地通过人机协作产生，战略性地在看似真实可信的叙述中嵌入不实信息。这类混合真伪内容构成现实且严重的威胁，却在现有基准中严重缺失。为此，我们提出MANYFAKE，一个包含6,798篇通过多种策略驱动提示生成的合成假新闻数据集，覆盖多种假新闻构建与优化方式。利用该基准，我们评估了多种前沿假新闻检测模型。结果表明，即使具备高级推理能力的模型在完全虚构内容上已接近性能饱和，但在面对细微、优化过的虚假信息与真实内容交织的情况时仍表现脆弱。
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs) have enabled the large-scale generation of highly fluent and deceptive news-like content. While prior work has often treated fake news detection as a binary classification problem, modern fake news increasingly arises through human-AI collaboration, where strategic inaccuracies are embedded within otherwise accurate and credible narratives. These mixed-truth cases represent a realistic and consequential threat, yet they remain underrepresented in existing benchmarks. To address this gap, we introduce MANYFAKE, a synthetic benchmark containing 6,798 fake news articles generated through multiple strategy-driven prompting pipelines that capture many ways fake news can be constructed and refined. Using this benchmark, we evaluate a range of state-of-the-art fake news detectors. Our results show that even advanced reasoning-enabled models approach saturation on fully fabricated stories, but remain brittle when falsehoods are subtle, optimized, and interwoven with accurate information.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。