提出多维可信度评估框架，用AI辅助人工核证者高效筛选待查信息。
Exploring Multidimensional Checkworthiness: Designing AI-assisted Claim Prioritization for Human Fact-checkers
- 将核证优先级视为多维度信息检索问题，挖掘影响判断的多种因素。
- 16名专业核证者揭示分层优先策略，证实多维可信度的复杂性。
- 设计支持系统并融合大模型，为核证流程提供可落地的智能辅助方案。
面对网络上海量潜在虚假信息，合理分配有限的人力资源进行核证至关重要。本文将核证优先级视为信息检索任务：正如多维相关性受多种因素影响，可信度同样具有多面性、主观性甚至个性化特征。本研究通过设计研究与混合方法评估，开发了一个AI辅助的核证优先级原型系统，探究核证者如何利用多维可信度因子进行优先排序，并识别其实际需求与设计空间。16名专业核证者参与实验，结果揭示出一种隐含的分层优先策略，揭示了核证工作流中尚未被充分关注的环节，提出了改进多维可信度判断及结合大模型优化核证流程的具体设计建议。
原文摘要 · Abstract (English)
Given the volume of potentially false claims online, claim prioritization is essential in allocating limited human resources available for fact-checking. In this study, we perceive claim prioritization as an information retrieval (IR) task: just as multidimensional IR relevance, with many factors influencing which search results a user deems relevant, checkworthiness is also multi-faceted, subjective, and even personal, with many factors influencing how fact-checkers triage and select which claims to check. Our study investigates both the multidimensional nature of checkworthiness and effective tool support to assist fact-checkers in claim prioritization. Methodologically, we pursue Research through Design combined with mixed-method evaluation. Specifically, we develop an AI-assisted claim prioritization prototype as a probe to explore how fact-checkers use multidimensional checkworthy factors to prioritize claims, simultaneously probing fact-checker needs and exploring the design space to meet those needs. With 16 professional fact-checkers participating in our study, we uncover a hierarchical prioritization strategy fact-checkers implicitly use, revealing an underexplored aspect of their workflow, with actionable design recommendations for improving claim triage across multidimensional checkworthiness and tailoring this process with LLM integration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。