用大模型加速健康谣言治理，提升内容审核的准确与及时性
Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation
- 引入大模型分三阶段评估内容：相关性、正确性、帮助性
- 实测显示模型在准确性、帮助性上优于人类标注者
- 解决用户误把文笔好当真实的问题，适合平台治理团队参考
X（原推特）的社区笔记系统允许用户标记误导性内容并附加解释，但对3.08万条健康类笔记的分析显示，其帮助性评定存在17.6小时的中位延迟。为应对信息传播高峰，我们提出CrowdNotes+——一种基于大模型的统一框架，通过证据增强与效用引导双模式，结合三层评估机制（相关性、正确性、帮助性）实现快速响应。我们构建了包含1200条健康笔记的HealthNotes基准数据集，并训练专用帮助性判断模型。分析发现，当前众包治理中，用户常将语言流畅度误认为事实准确性。实验对比15种代表性LLM表明，CrowdNotes+在正确性、帮助性和证据利用率上均显著超越人工标注者。
原文摘要 · Abstract (English)
Community Notes, the crowd-sourced misinformation governance system on X (formerly Twitter), allows users to flag misleading posts, attach contextual notes, and rate the notes' helpfulness. However, our empirical analysis of 30.8K health-related notes reveals substantial latency, with a median delay of 17.6 hours before notes receive a helpfulness status. To improve responsiveness during real-world misinformation surges, we propose CrowdNotes+, a unified LLM-based framework that augments Community Notes for faster and more reliable health misinformation governance. CrowdNotes+ integrates two modes: (1) evidence-grounded note augmentation and (2) utility-guided note automation, supported by a hierarchical three-stage evaluation of relevance, correctness, and helpfulness. We instantiate the framework with HealthNotes, a benchmark of 1.2K health notes annotated for helpfulness, and a fine-tuned helpfulness judge. Our analysis first uncovers a key loophole in current crowd-sourced governance: voters frequently conflate stylistic fluency with factual accuracy. Addressing this via our hierarchical evaluation, experiments across 15 representative LLMs demonstrate that CrowdNotes+ significantly outperforms human contributors in note correctness, helpfulness, and evidence utility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。