攻击医疗多模态生成系统,用隐蔽误导信息破坏诊断准确性
Knowledge Poisoning Attacks on Medical Multi-Modal Retrieval-Augmented Generation

- 通过视觉触发+文本扰动,在不依赖用户查询的前提下污染知识库
- 在5个大模型和数据集上均生成临床看似合理但错误的诊断结果
- 适合关注医疗AI安全、对抗攻击的研究者与开发者
检索增强生成(RAG)被广泛用于提升医疗大模型性能,通过引入专家多模态知识来增强生成内容。然而,底层检索数据库可能天然或被恶意注入对抗性知识,从而干扰模型输出并损害系统可靠性。现有研究虽已探索医疗RAG中的知识中毒攻击,但大多假设攻击者事先知晓用户查询,这在实际部署中不现实,严重限制了其应用价值。本文提出M³Att框架,针对仅掌握数据库分布知识的攻击场景,设计了一种无需依赖查询的多模态知识中毒方法。核心思想是在文本中注入隐蔽误导信息,同时利用配对视觉数据作为与查询无关的触发信号以引导错误检索。我们提出统一框架,对视觉输入施加不可察觉的扰动,操纵检索概率;由于大模型本身具备医学先验知识,直接伪造事实的内容常会被自纠正,因此我们利用医学诊断固有的模糊性,设计了可规避自修正的隐蔽误导策略,显著降低诊断准确率。在五个大模型和数据集上的实验表明,M³Att能持续生成看似合理但错误的临床结论。代码已开源:https://github.com/ypr17/M3Att。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) is a widely adopted paradigm for enhancing LLMs in medical applications by incorporating expert multimodal knowledge during generation. However, the underlying retrieval databases may naturally contain, or be intentionally injected with, adversarial knowledge, which can perturb model outputs and undermine system reliability. To investigate this risk, prior studies have explored knowledge poisoning attacks in medical RAG systems. Nevertheless, most of them rely on the strong assumption that adversaries possess prior knowledge of user queries, which is unrealistic in deployments and substantially limits their practical applicability. In this paper, we propose M\textsuperscript{3}Att, a knowledge-poisoning framework designed for medical multimodal RAG systems, assuming only limited distribution knowledge of the underlying database. Our core idea is to inject covert misinformation into textual data while using paired visual data as a query-agnostic trigger to promote retrieval. We first propose a unified framework that introduces imperceptible perturbations to visual inputs to manipulate retrieval probabilities. Besides, due to the prior medical knowledge in LLMs, naively poisoned medical content with explicit factual errors can be corrected during generation. Thus, we leverage the inherent ambiguity of medical diagnosis and design a covert misinformation injection strategy that degrades diagnostic accuracy while evading model self-correction. Experiments on five LLMs and datasets demonstrate that M\textsuperscript{3}Att consistently produces clinically plausible yet incorrect generations. Codes: https://github.com/ypr17/M3Att.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。