构建首个覆盖58种低资源语言的虚假内容检测基准，填补多语言假信息防御空白。
BLUFF: Benchmarking the Detection of False and Synthetic Content across 58 Low-Resource Languages
- 构建跨79种语言的多类型虚假/合成内容数据集，含人工与大模型生成内容
- 在低资源语言上检测模型性能下降达25.3% F1，凸显检测不平等
- 提供开源工具与评估框架，助力公平化虚假信息检测研究
多语言虚假信息威胁全球信息完整性，但现有检测基准仍局限于英语或少数高资源语言，导致低资源语言社区缺乏有效防御工具。我们提出BLUFF，一个涵盖79种语言、超过20.2万样本的综合性虚假与合成内容检测基准，整合了12.2万+样本的人工撰写事实核查内容（覆盖57种语言）和7.9万+样本的大模型生成内容（覆盖71种语言）。该数据集独特覆盖高资源（20种）与低资源（59种）语言，弥补多语言虚假内容检测研究的关键空白。数据集包含四类内容（人工撰写、大模型生成、大模型翻译、人机混合文本）、双向翻译（英语↔目标语言）、39种文本修改技术（36种假新闻操纵策略、3种真实新闻AI编辑方法），以及使用19种不同大模型生成的不同强度修改内容。我们提出AXL-CoI（对抗性跨语言多智能体交互链）框架用于受控假/真新闻生成，并配套mPURIFY质量过滤流程确保数据完整性。实验表明，当前顶尖检测器在低资源语言上的F1分数最高下降25.3%。BLUFF为研究社区提供多语言基准、全面语言导向评估、完整文档及开源工具，推动公平虚假信息检测发展。数据集与代码已公开：https://jsl5710.github.io/BLUFF/
原文摘要 · Abstract (English)
Multilingual falsehoods threaten information integrity worldwide, yet detection benchmarks remain confined to English or a few high-resource languages, leaving low-resource linguistic communities without robust defense tools. We introduce BLUFF, a comprehensive benchmark for detecting false and synthetic content, spanning 79 languages with over 202K samples, combining human-written fact-checked content (122K+ samples across 57 languages) and LLM-generated content (79K+ samples across 71 languages). BLUFF uniquely covers both high-resource "big-head" (20) and low-resource "long-tail" (59) languages, addressing critical gaps in multilingual research on detecting false and synthetic content. Our dataset features four content types (human-written, LLM-generated, LLM-translated, and hybrid human-LLM text), bidirectional translation (English$\leftrightarrow$X), 39 textual modification techniques (36 manipulation tactics for fake news, 3 AI-editing strategies for real news), and varying edit intensities generated using 19 diverse LLMs. We present AXL-CoI (Adversarial Cross-Lingual Agentic Chainof-Interactions), a novel multi-agentic framework for controlled fake/real news generation, paired with mPURIFY, a quality filtering pipeline ensuring dataset integrity. Experiments reveal state-of-theart detectors suffer up to 25.3% F1 degradation on low-resource versus high-resource languages. BLUFF provides the research community with a multilingual benchmark, extensive linguistic-oriented benchmark evaluation, comprehensive documentation, and opensource tools to advance equitable falsehood detection. Dataset and code are available at: https://jsl5710.github.io/BLUFF/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。