测试大模型在生物医学问答中对抗错误信息的能力，发现小模型也能通过外部知识提升表现。
LLM Robustness Against Misinformation in Biomedical Question Answering
- 用外部知识增强大模型问答，防止虚构答案
- 小模型（如Llama 3.1）在正确知识下准确率达80.2%
- 模型越强越易被恶意信息误导，鲁棒性评估需多维度
检索增强生成（RAG）通过引入外部知识源来减少大语言模型（LLM）在问答中的虚构现象。然而，错误信息注入可能导致模型生成错误答案。本文评估了四种LLM（Gemma 2、GPT-4o-mini、Llama 3.1、Mixtral）在生物医学问答中对误导信息的鲁棒性，涵盖三种场景：无上下文的原始回答（准确率0.651）、提供正确上下文的完美增强（准确率0.802）、以及注入错误上下文的提示攻击。结果表明，Llama 3.1（70B参数）在原始和完美增强场景中表现最佳。在完美RAG下，各模型性能差距缩小，显示其可缓解规模带来的差异。进一步实验中，使用相同四类模型生成恶意上下文进行攻击，发现Llama 3.1作为攻击者时导致目标模型准确率下降最高达0.48（原始）和0.63（完美增强）。不同评估指标下的鲁棒性排名不一致，凸显评估复杂性。
原文摘要 · Abstract (English)
The retrieval-augmented generation (RAG) approach is used to reduce the confabulation of large language models (LLMs) for question answering by retrieving and providing additional context coming from external knowledge sources (e.g., by adding the context to the prompt). However, injecting incorrect information can mislead the LLM to generate an incorrect answer. In this paper, we evaluate the effectiveness and robustness of four LLMs against misinformation - Gemma 2, GPT-4o-mini, Llama~3.1, and Mixtral - in answering biomedical questions. We assess the answer accuracy on yes-no and free-form questions in three scenarios: vanilla LLM answers (no context is provided), "perfect" augmented generation (correct context is provided), and prompt-injection attacks (incorrect context is provided). Our results show that Llama 3.1 (70B parameters) achieves the highest accuracy in both vanilla (0.651) and "perfect" RAG (0.802) scenarios. However, the accuracy gap between the models almost disappears with "perfect" RAG, suggesting its potential to mitigate the LLM's size-related effectiveness differences. We further evaluate the ability of the LLMs to generate malicious context on one hand and the LLM's robustness against prompt-injection attacks on the other hand, using metrics such as attack success rate (ASR), accuracy under attack, and accuracy drop. As adversaries, we use the same four LLMs (Gemma 2, GPT-4o-mini, Llama 3.1, and Mixtral) to generate incorrect context that is injected in the target model's prompt. Interestingly, Llama is shown to be the most effective adversary, causing accuracy drops of up to 0.48 for vanilla answers and 0.63 for "perfect" RAG across target models. Our analysis reveals that robustness rankings vary depending on the evaluation measure, highlighting the complexity of assessing LLM resilience to adversarial attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。