通过引入反论点等信息，提升大模型识别逻辑谬误能力
Large Language Models Are Better Logical Fallacy Reasoners with Counterargument, Explanation, and Goal-Aware Prompt Formulation
- 用反论点、解释和目标信息增强输入，帮助模型理解论证上下文
- 零样本下F1最高提升0.60，微调模型提升0.45
- 适合需要高精度谬误检测的研究者与内容审核应用
大语言模型在处理复杂语言方面取得显著进展，但准确检测逻辑谬误仍具挑战。本文提出一种新颖有效的提示构建方法，适用于监督（微调）与非监督（零样本）场景。该方法通过在输入中融入反论点、解释和论证目标等隐含上下文信息，并评估其在论证中的有效性，再根据置信度对这些查询进行排序以辅助分类。我们在五个领域共29种谬误类型的数据集上进行了评估，使用GPT与LLaMA系列模型。结果表明，相比现有最优模型，零样本设置下F1得分最高提升0.60，微调模型提升达0.45。大量分析进一步揭示了该方法的优势机制。
原文摘要 · Abstract (English)
The advancement of Large Language Models (LLMs) has greatly improved our ability to process complex language. However, accurately detecting logical fallacies remains a significant challenge. This study presents a novel and effective prompt formulation approach for logical fallacy detection, applicable in both supervised (fine-tuned) and unsupervised (zero-shot) settings. Our method enriches input text incorporating implicit contextual information -- counterarguments, explanations, and goals -- which we query for validity within the context of the argument. We then rank these queries based on confidence scores to inform classification. We evaluate our approach across multiple datasets from 5 domains, covering 29 distinct fallacy types, using models from the GPT and LLaMA series. The results show substantial improvements over state-of-the-art models, with F1 score increases of up to 0.60 in zero-shot settings and up to 0.45 in fine-tuned models. Extensive analyses further illustrate why and how our method excels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。