无需标注数据，自动优化推文表述以提升事实核查准确率
Self-Adaptive Paraphrasing and Preference Learning for Improved Claim Verifiability
- 用生成模型迭代优化推文表述，使其更易被核查
- 在假消息上表现优于所有基线模型
- 仅需黑箱核查模型与语言模型，适合无标注数据场景
在事实核查中，陈述的结构和措辞显著影响模型判断准确性。社交媒体内容通常包含噪声，难以直接用于验证系统，需先提取出清晰陈述。已有方法虽能提取人类认为可核查的陈述，但存在两个问题：一是格式未必适合核查模型，二是需要标注数据训练提取任务。本文提出一种无需标注数据的自适应方法，仅依赖黑箱事实核查模型与生成语言模型。给定一条推文，通过迭代优化语言模型生成能提升核查模型性能的陈述改写版本，并基于偏好对使用直接偏好优化进行对齐。实验表明，该方法生成的陈述比原始社交媒体表述更具可核查性，且性能媲美先进基线；在被驳回的陈述上，始终优于所有基线。
原文摘要 · Abstract (English)
In fact-checking, structure and phrasing of claims critically influence a model's ability to predict verdicts accurately. Social media content in particular rarely serves as optimal input for verification systems, which necessitates pre-processing to extract the claim from noisy context before fact checking. Prior work suggests extracting a claim representation that humans find to be checkworthy and verifiable. This has two limitations: (1) the format may not be optimal for a fact-checking model, and (2), it requires annotated data to learn the extraction task from. We address both issues and propose a method to extract claims that is not reliant on labeled training data. Instead, our self-adaptive approach only requires a black-box fact checking model and a generative language model (LM). Given a tweet, we iteratively optimize the LM to generate a claim paraphrase that increases the performance of a fact checking model. By learning from preference pairs, we align the LM to the fact checker using direct preference optimization. We show that this novel setup extracts a claim paraphrase that is more verifiable than their original social media formulations, and is on par with competitive baselines. For refuted claims, our method consistently outperforms all baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。