首个细粒度假新闻片段识别数据集,可定位并分类句子中的虚假信息。
MisSpans: Fine-Grained False Span Identification in Cross-Domain Fake News
- 构建多领域真实与虚假新闻对,支持段落级假信息定位与分类
- 15个大模型测试显示细粒度识别难度高,模型能力受规模与推理机制影响
- 提供可解释的虚假段落分析,适合研究假新闻检测与可解释AI
在线虚假信息日益泛滥,但现有评估大多以整句或段落为单位,采用粗粒度二元标签,掩盖了真伪信息常共存于单一句子的事实。此类简化也限制了可解释性:全局解释无法定位具体误导段落,也无法区分虚假类型(如扭曲或虚构)。为此,我们提出MisSpans,首个跨领域的、人工标注的细粒度假信息段落检测与分析基准,包含成对的真实与虚假新闻故事。MisSpans定义三个互补任务:MisSpansIdentity用于精确定位句子中的虚假段落,MisSpansType用于按虚假类型分类,MisSpansExplanation用于基于识别段落生成可解释理由。三项任务共同实现细粒度定位、精细化表征及可操作解释。专家标注遵循标准化指南与一致性检查,达成高一致性。我们在零样本和单样本设置下评估15个代表性大语言模型(含增强推理与非推理版本),结果揭示细粒度假信息识别的挑战性,并凸显模型表现受模型规模、推理能力及领域文本特征等多重因素交互影响。项目将开源于https://github.com/lzw108/MisSpans。
原文摘要 · Abstract (English)
Online misinformation is increasingly pervasive, yet most existing benchmarks and methods evaluate veracity at the level of whole claims or paragraphs using coarse binary labels, obscuring how true and false details often co-exist within single sentences. These simplifications also limit interpretability: global explanations cannot identify which specific segments are misleading or differentiate how a detail is false (e.g., distorted vs. fabricated). To address these gaps, we introduce MisSpans, the first multi-domain, human-annotated benchmark for span-level misinformation detection and analysis, consisting of paired real and fake news stories. MisSpans defines three complementary tasks: MisSpansIdentity for pinpointing false spans within sentences, MisSpansType for categorising false spans by misinformation type, and MisSpansExplanation for providing rationales grounded in identified spans. Together, these tasks enable fine-grained localisation, nuanced characterisation beyond true/false and actionable explanations. Expert annotators were guided by standardised guidelines and consistency checks, leading to high inter-annotator agreement. We evaluate 15 representative LLMs, including reasoning-enhanced and non-reasoning variants, under zero-shot and one-shot settings. Results reveal the challenging nature of fine-grained misinformation identification and analysis, and highlight the need for a deeper understanding of how performance may be influenced by multiple interacting factors, including model size and reasoning capabilities, along with domain-specific textual features. This project will be available at https://github.com/lzw108/MisSpans.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。