用分步指令+知识图谱提升大模型逻辑谬误识别能力
Follow My Lead: Logical Fallacy Classification with Knowledge-Augmented LLMs
- 将谬误分类拆解为一系列简单二元问题，引导模型逐步推理
- 结合关系知识图谱验证结果，准确率显著提升
- 方法透明可解释，适合需要可靠推理的场景
大语言模型存在关键推理缺陷，易产生幻觉且谬误识别准确率低，这源于其默认的快速直觉式系统1处理模式。可靠推理需依赖费力的系统2方式（Kahneman, 2011；Li et al., 2025）。由于完整系统2训练成本过高，本文提出低成本指令干预方案：构建新型分步指令数据集，将谬误分类分解为一系列原子化程序步骤（简单二元问题），并引入最终验证环节，让模型查阅相关谬误的关系知识图谱。该程序化、规则化方法显著提升大模型在逻辑谬误分类上的表现，同时增强决策过程透明度，为神经符号架构解决大模型推理缺陷提供了可行路径。
原文摘要 · Abstract (English)
Large Language Models (LLMs) suffer from critical reasoning gaps, including a tendency to hallucinate and poor accuracy in classifying logical fallacies. This limitation stems from their default System 1 processing, which is fast and intuitive, whereas reliable reasoning requires the deliberate, effortful System 2 approach (Kahneman, 2011; Li et al., 2025). Since full System 2 training is often prohibitively expensive, we explore a low-cost, instruction-based intervention to bridge this gap. Our methodology introduces a novel stepwise instruction dataset that decomposes fallacy classification into a series of atomic procedural steps (simple binary questions). We further augment this with a final verification step where models consult a relational knowledge graph of related fallacies. This procedural, rule-based intervention yields a significant improvement in LLM logical fallacy classification. Crucially, the approach also provides enhanced transparency into the LLMs' decision-making, highlighting a practical pathway for Neuro-symbolic architectures to address LLM reasoning deficits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。