让大模型按需生成并筛选常识知识，提升自然语言推理准确率。
Commonsense on Demand: Generating and Selectively Integrating Commonsense Knowledge for Natural Language Inference
- 用大模型自动生成常识命题，再通过自身判断其真实性
- 混合筛选策略使推理准确率提升3.87%至8.5%
- 特别缓解模型对中立类的偏见，适合高精度推理场景
自然语言推理（NLI）判断前提是否蕴含、矛盾或与假设无关。该任务常被视作模拟人类推理，其中常识知识起关键作用。本文研究大语言模型（LLM）能否可靠生成适用于NLI的事实性常识公理，并在SNLI和ANLI基准上使用Llama-3.1-70B与gpt-oss-120b进行评估。由于常识命题缺乏显式文本依据，传统事实性指标不适用。因此提出一种无需参考的评估方法，采用大模型作为裁判（LLM-as-Judge）。评估显示模型间差距显著：gpt-oss-120b生成的命题大多正确，而Llama生成的错误多于正确。进一步测试三种提示流程：直接推理、加入生成常识的增强推理，以及基于判断事实性的选择性融合。混合方法在所有测试配置中均带来3.87%–8.5%的一致准确率提升。有针对性地引入常识知识还帮助模型克服对中立类的倾向，提供必要的现实背景。
原文摘要 · Abstract (English)
Natural Language Inference (NLI) determines whether a premise entails, contradicts, or is neutral with respect to a hypothesis. The task is often framed as emulating human inference, in which commonsense knowledge plays a major role. This study examines whether Large Language Models (LLMs) can reliably generate factual commonsense axioms for NLI, and evaluates their utility on the SNLI and ANLI benchmarks using Llama-3.1-70B and gpt-oss-120b. Because commonsense axioms lack explicit textual references, standard factuality metrics are ill-suited to their evaluation. We therefore introduce a reference-free method using an LLM-as-Judge framework. The evaluation reveals a substantial gap between models: gpt-oss-120b generates predominantly accurate axioms, whereas Llama produces more incorrect than correct ones. We further evaluate three prompting pipelines: direct inference, inference augmented with generated commonsense axioms, and a hybrid approach that selectively incorporates highly factual axioms based on judged factuality. The hybrid approach yields consistent accuracy gains of 3.87%-8.5% across tested configurations. Targeted commonsense knowledge also helps models overcome a bias toward the Neutral class by providing essential real-world context.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。