大模型在反事实推理中难以融合新知识与自身知识。
Can LLMs Reconcile Knowledge Conflicts in Counterfactual Reasoning
- 通过反事实推理任务测试模型融合新旧知识的能力
- 多数模型仍依赖参数化知识，无法有效整合上下文信息
- 微调可能破坏原有知识，适合研究知识冲突的学者参考
大型语言模型在参数中存储了大量世界知识，使其在诸多知识密集型任务中表现优异。然而，在新场景下部署时，模型常需将参数知识与新信息融合。本文从反事实推理视角，通过合成与真实多跳推理实验，发现大模型普遍难以进行反事实推理，往往仅依赖其参数化知识。此外，简单的后处理微调难以培养反事实推理能力，且常导致原有参数知识退化。本工作揭示了当前大模型在新情境下重新利用参数知识的重要局限。
原文摘要 · Abstract (English)
Large Language Models have been shown to contain extensive world knowledge in their parameters, enabling impressive performance on many knowledge intensive tasks. However, when deployed in novel settings, LLMs often encounter situations where they must integrate parametric knowledge with new or unfamiliar information. In this work, we explore whether LLMs can combine knowledge in-context with their parametric knowledge through the lens of counterfactual reasoning. Through synthetic and real experiments in multi-hop reasoning problems, we show that LLMs generally struggle with counterfactual reasoning, often resorting to exclusively using their parametric knowledge. Moreover, we show that simple post-hoc finetuning can struggle to instill counterfactual reasoning ability -- often leading to degradation in stored parametric knowledge. Ultimately, our work reveals important limitations of current LLM's abilities to re-purpose parametric knowledge in novel settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。