用指令模型实现零样本/少样本声明匹配，提升自动验真效率
Zero-shot and Few-shot Learning with Instruction-following LLMs for Claim Matching in Automated Fact-checking
- 将声明匹配视为二分类任务，利用指令型大模型与提示模板
- 在新数据集ClaimMatch上验证，少样本下准确率超80%
- 适合需快速部署的自动验真系统开发者
声明匹配（CM）任务可通过整合可由同一事实验证的声明，助力自动化验真流程。本文首次探索该任务的零样本与少样本学习方法。我们将CM建模为二分类问题，测试了GPT-3.5-turbo、Gemini-1.5-flash、Mistral-7B-Instruct和Llama-3-8B-Instruct等指令型大模型，并研究不同提示模板效果。我们构建了新的CM数据集ClaimMatch，将在论文接受后公开。实验表明，通过借鉴自然语言推理或释义检测等成熟任务，可有效解决CM问题。我们还提出一个适用于不同文本长度的CM处理流水线。
原文摘要 · Abstract (English)
The claim matching (CM) task can benefit an automated fact-checking pipeline by putting together claims that can be resolved with the same fact-check. In this work, we are the first to explore zero-shot and few-shot learning approaches to the task. We consider CM as a binary classification task and experiment with a set of instruction-following large language models (GPT-3.5-turbo, Gemini-1.5-flash, Mistral-7B-Instruct, and Llama-3-8B-Instruct), investigating prompt templates. We introduce a new CM dataset, ClaimMatch, which will be released upon acceptance. We put LLMs to the test in the CM task and find that it can be tackled by leveraging more mature yet similar tasks such as natural language inference or paraphrase detection. We also propose a pipeline for CM, which we evaluate on texts of different lengths.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。