研究大模型在问答中受假信息影响时的知识漂移现象
Understanding Knowledge Drift in LLMs through Misinformation
- 通过问答场景测试大模型对假信息的敏感性
- 错误回答时不确定性上升56.6%，重复暴露可使不确定性下降52.8%
- 揭示大模型易被误导的机制，适合安全与可信AI研究者
大型语言模型（LLMs）已广泛应用于各类场景，但其可靠性在面对虚假信息时尤为关键。本文重点分析了先进LLMs在问答任务中遭遇假信息时对事实性错误的敏感性，该问题可能导致我们称为‘知识漂移’的现象，严重削弱模型可信度。我们基于熵、困惑度和词元概率等指标评估模型响应的事实性和不确定性。实验表明，当问题因假信息导致回答错误时，模型的不确定性最高可上升56.6%；而反复接触同一假信息后，不确定性又可下降52.8%（相较于未污染提示下的回答），暗示模型信念可能被操纵并偏离原始知识。这些发现揭示了大模型对对抗输入的脆弱性与鲁棒性，为提升各领域应用的可靠性提供了方向。代码已开源。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have revolutionized numerous applications, making them an integral part of our digital ecosystem. However, their reliability becomes critical, especially when these models are exposed to misinformation. We primarily analyze the susceptibility of state-of-the-art LLMs to factual inaccuracies when they encounter false information in a QnA scenario, an issue that can lead to a phenomenon we refer to as *knowledge drift*, which significantly undermines the trustworthiness of these models. We evaluate the factuality and the uncertainty of the models' responses relying on Entropy, Perplexity, and Token Probability metrics. Our experiments reveal that an LLM's uncertainty can increase up to 56.6% when the question is answered incorrectly due to the exposure to false information. At the same time, repeated exposure to the same false information can decrease the models uncertainty again (-52.8% w.r.t. the answers on the untainted prompts), potentially manipulating the underlying model's beliefs and introducing a drift from its original knowledge. These findings provide insights into LLMs' robustness and vulnerability to adversarial inputs, paving the way for developing more reliable LLM applications across various domains. The code is available at https://github.com/afastowski/knowledge_drift.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。