arXiv:2602.10935cs.HCcs.AI2026-02

用户最常查的是当下可见的简单陈述,而非复杂或主观判断。

What do people want to fact-check?

  • 分析近2500条用户提交的待查陈述,按五个维度分类,绘制公众验证需求图谱。
  • 约四分之一请求无法实证验证,涉及道德判断、预测和主观评价。
  • 现有评测数据集与真实用户需求差异显著,揭示当前系统设计偏差。

关于虚假信息的研究几乎只关注供给端,探讨错误信息的传播、制造者及纠错效果。但一个基本的需求侧问题仍未解答:当普通人可自由核查任何内容时,他们实际关心什么?本研究通过分析457名参与者提交的近2500条开放式陈述,首次提供了大规模证据。每条声明按五个语义维度(领域、认知形式、可验证性、目标实体、时间参照)分类,构建出公众验证需求的行为地图。三大发现突出:第一,用户涉猎广泛话题,但认知模式狭窄,主要提交关于当下可观测事实的简单描述;第二,约四分之一请求涉及无法实证解决的内容,包括道德判断、推测性预测和主观评价,暴露用户需求与工具能力间的系统性错配;第三,与FEVER基准数据集对比显示,所有五个维度均存在显著结构差异,表明标准评估数据集所模拟的陈述环境并非真实世界验证需求的反映。这些结果将事实核查重新定义为需求驱动的问题,并指出现有AI系统与评测基准在应对人们真实不确定性上的严重脱节。

原文摘要 · Abstract (English)

Research on misinformation has focused almost exclusively on supply, asking what falsehoods circulate, who produces them, and whether corrections work. A basic demand-side question remains unanswered. When ordinary people can fact-check anything they want, what do they actually ask about? We provide the first large-scale evidence on this question by analyzing close to 2{,}500 statements submitted by 457 participants to an open-ended AI fact-checking system. Each claim is classified along five semantic dimensions (domain, epistemic form, verifiability, target entity, and temporal reference), producing a behavioral map of public verification demand. Three findings stand out. First, users range widely across topics but default to a narrow epistemic repertoire, overwhelmingly submitting simple descriptive claims about present-day observables. Second, roughly one in four requests concerns statements that cannot be empirically resolved, including moral judgments, speculative predictions, and subjective evaluations, revealing a systematic mismatch between what users seek from fact-checking tools and what such tools can deliver. Third, comparison with the FEVER benchmark dataset exposes sharp structural divergences across all five dimensions, indicating that standard evaluation corpora encode a synthetic claim environment that does not resemble real-world verification needs. These results reframe fact-checking as a demand-driven problem and identify where current AI systems and benchmarks are misaligned with the uncertainty people actually experience.

事实核查用户行为数据偏差AI评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。