提出可检测AI信息清洗的开源工具,防止谣言变事实。
FACTWASH: Catching AI Rewrites That Wash Hearsay into Fact
- 用关键词列表和证据标记实现确定性检测,不依赖大模型判断。
- 在未调优文本上达0.91 F1,对模糊表达召回率提升17点。
- 适合关注AI生成内容可信度的研究者与开发者使用。
AI系统持续改写信息:对话变记忆,文档变回答。改写可能保留陈述却抹去可验证性依据——谁说的、有多确定、何时成立。我们称此为‘事实清洗’(factwashing),并发布factwash,一个开源的写入期检测机制,通过命名标记和证据而非大模型判断来确定性识别问题。该工具解答了一个实际问题:何时只需简单检查,何时需模型介入?关键在于属性是否具有有限表面线索集合。显式否定线索接近可枚举,仅靠词表即可完成,未调优文本上达到0.91 F1。而缓和语义与归属信息存在开放实现,词汇量达瓶颈时召回率仅半,引入单问题大模型见证可分别提升17和15点召回率,精度不变。部署后该见证仅降低判断结果,不扩大覆盖,反而提升精度。我们在105,596条独立标注句中评估线索检测效果。对记忆写入的盲标语料分析显示,55%的错误写入来自对话传闻,7%来自商务邮件(p < 0.001),因此首个部署问题不是选哪个检测器,而是该失败是否真实发生。在未修改的mem0 2.0.7上,该门控机制能识别出8条中有5条的缓和性传闻写入。
原文摘要 · Abstract (English)
AI systems rewrite information constantly: conversations become stored memories, documents become answers. The rewrite can keep a claim while washing away what made it checkable, who said it, how sure they were, when it held. We call that failure factwashing, and release factwash, an open-source write-time gate that catches it deterministically, with named flags and evidence rather than an LLM judge. Building it answers a practical question: when does a cheap check suffice, and when do you need a model? What decides is whether the property has a bounded surface-cue inventory. Explicit negation cues are close to enumerable, so a word list finishes and transfers, reaching 0.91 F1 on untuned text. Hedging and attribution have open-ended realizations, so vocabulary plateaus near half recall, and a one-question LLM witness recovers +17 and +15 points of cue-detection recall at equal precision. Deployed, that witness may only lower a verdict, so it buys precision rather than coverage. We measure cue detection on 105,596 independently annotated sentences. A blind-labelled corpus of memory writes then locates the failure: 55% of bad writes in conversational hearsay, 7% in business email (p < 0.001), so the first deployment question is not which detector to use but whether the failure occurs at all. On unmodified mem0 2.0.7, the gate flags 5 of 8 hedged-hearsay writes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。