用缺失信息定位增强社区注释生成,提升覆盖率与质量。
GitSearch: Enhancing Community Notes Generation with Gap-Informed Targeted Search
- 将感知质量缺口作为信号,动态检索补全信息
- 覆盖率达99%,近乎翻倍于现有方法
- 优于人工注释,适合大规模内容审核场景
基于社区的监管为规模化事实核查提供了替代方案,但面临结构性挑战,现有基于AI的方法在冷启动场景下表现不佳。为此,我们提出GitSearch(Gap-Informed Targeted Search)框架,将人类感知的质量缺口(如信息缺失)视为首要信号。该框架包含三阶段流程:识别信息缺陷、实时执行针对性网络检索以填补空白、合成符合平台规范的注释。为支持评估,我们构建了PolBench基准数据集,包含78,698条美国政治推文及其关联的社区注释。实验表明,GitSearch实现99%的覆盖度,几乎达到当前最优水平的两倍;其注释在帮助性上超越人工撰写,胜率高达69%,得分3.87对比人工3.36,验证了其在规模与质量间的有效平衡。
原文摘要 · Abstract (English)
Community-based moderation offers a scalable alternative to centralized fact-checking, yet it faces significant structural challenges, and existing AI-based methods fail in "cold start" scenarios. To tackle these challenges, we introduce GitSearch (Gap-Informed Targeted Search), a framework that treats human-perceived quality gaps, such as missing context, etc., as first-class signals. GitSearch has a three-stage pipeline: identifying information deficits, executing real-time targeted web-retrieval to resolve them, and synthesizing platform-compliant notes. To facilitate evaluation, we present PolBench, a benchmark of 78,698 U.S. political tweets with their associated Community Notes. We find GitSearch achieves 99% coverage, almost doubling coverage over the state-of-the-art. GitSearch surpasses human-authored helpful notes with a 69% win rate and superior helpfulness scores (3.87 vs. 3.36), demonstrating retrieval effectiveness that balanced the trade-off between scale and quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。