用大模型零样本实现可解释的事实验证，无需标注数据。
Zero-Shot Fact Verification via Natural Logic and Large Language Models
- 利用指令微调的大模型直接做事实验证，不依赖自然逻辑标注数据。
- 在真实和人工命题上平均准确率比最佳基线高8.96个百分点。
- 适合跨领域、无训练数据的事实验证场景，尤其对多语言任务有效。
近年来，基于自然逻辑的事实验证系统通过集合论操作将陈述与证据对齐，提升了可解释性，提供了可信的推理依据。然而，这些系统通常需要大量标注了自然逻辑的数据进行训练。为解决此问题，我们提出一种零样本方法，充分利用指令微调大语言模型的泛化能力。为全面评估该方法及其他验证系统的零样本性能，我们在人工和真实世界命题上进行测试，涵盖多语言数据集。实验分为两种设置：第一，在零样本泛化设置中，我们的方法优于未专门在自然逻辑数据上训练的其他系统，平均准确率提升8.96点；第二，在零样本迁移设置中，发现现有自然逻辑训练系统在跨领域泛化上表现不佳，而我们的方法在所有含真实命题的数据集上均表现更优。
原文摘要 · Abstract (English)
The recent development of fact verification systems with natural logic has enhanced their explainability by aligning claims with evidence through set-theoretic operators, providing faithful justifications. Despite these advancements, such systems often rely on a large amount of training data annotated with natural logic. To address this issue, we propose a zero-shot method that utilizes the generalization capabilities of instruction-tuned large language models. To comprehensively assess the zero-shot capabilities of our method and other fact verification systems, we evaluate all models on both artificial and real-world claims, including multilingual datasets. We also compare our method against other fact verification systems in two setups. First, in the zero-shot generalization setup, we demonstrate that our approach outperforms other systems that were not specifically trained on natural logic data, achieving an average accuracy improvement of 8.96 points over the best-performing baseline. Second, in the zero-shot transfer setup, we show that current systems trained on natural logic data do not generalize well to other domains, and our method outperforms these systems across all datasets with real-world claims.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。