用对比学习提升事实核查的证据检索能力，让模型更懂隐含推理。
Contrastive Learning to Improve Retrieval for Real-world Fact Checking
- 基于多信号对比训练，优化检索模型对复杂线索的捕捉能力。
- 在AVerTeC上使事实判断准确率提升6%，跨数据集效果稳定。
- 适合需要深度推理的现实场景事实核查任务，尤其关注间接证据。
近期事实核查研究关注真实场景：模型需从网络检索证据以判断声明真伪。当前瓶颈在于如何获取相关证据——传统方法仅返回与声明直接相关的文档，但复杂声明常需推理推导。例如，关于疫苗成分的声明，其相关证据可能是疫苗研发过程的文档，即使未直接提及成分。我们提出对比事实核查重排序器（CFR），利用标注了子问题及答案的AVerTeC数据集，通过融合GPT-4蒸馏、子问题答案评估和数据集真实标签等多源信号，对Contriever进行对比目标微调。在检索与端到端真伪判断任务中评估，结果表明在AVerTeC上真伪分类准确率提升6%。同时，该优势可迁移至FEVER、ClaimDecomp、HotpotQA及合成数据集，验证其在需推理的检索任务中的泛化能力。
原文摘要 · Abstract (English)
Recent work on fact-checking addresses a realistic setting where models incorporate evidence retrieved from the web to decide the veracity of claims. A bottleneck in this pipeline is in retrieving relevant evidence: traditional methods may surface documents directly related to a claim, but fact-checking complex claims requires more inferences. For instance, a document about how a vaccine was developed is relevant to addressing claims about what it might contain, even if it does not address them directly. We present Contrastive Fact-Checking Reranker (CFR), an improved retriever for this setting. By leveraging the AVeriTeC dataset, which annotates subquestions for claims with human written answers from evidence documents, we fine-tune Contriever with a contrastive objective based on multiple training signals, including distillation from GPT-4, evaluating subquestion answers, and gold labels in the dataset. We evaluate our model on both retrieval and end-to-end veracity judgments about claims. On the AVeriTeC dataset, we find a 6\% improvement in veracity classification accuracy. We also show our gains can be transferred to FEVER, ClaimDecomp, HotpotQA, and a synthetic dataset requiring retrievers to make inferences.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。