为大模型医疗系统设计安全风险评估框架,识别攻击路径并指导防护。
Goal-Driven Risk Assessment for LLM-Powered Systems: A Healthcare Case Study
- 基于攻击树构建目标驱动的风险评估方法,明确攻击路径与前提条件。
- 在医疗大模型系统案例中验证,揭示新型攻击组合的潜在威胁。
- 适合关注大模型系统安全的设计者与安全研究人员参考。
将大语言模型(LLM)应用于医疗等关键领域虽带来显著优势,但其引入了新的安全挑战,特别是由对抗性模型、提示注入与传统网络攻击结合形成的攻击链。现有威胁建模方法虽能识别潜在威胁,但多为抽象描述,难以支撑有效的影响与发生概率评估,尤其在具有新型攻击面的复杂系统中。本文提出一种结构化的目标驱动风险评估方法,通过攻击树对威胁进行上下文化,明确攻击向量、前置条件及攻击路径。以基于大模型代理的医疗系统为例,验证该方法的有效性,整合当前主流的针对大模型攻击与传统攻击,并展示可复用于类似系统的攻击路径。该研究为大模型系统安全设计提供了结构化风险评估工具,推动了安全优先的设计实践。
原文摘要 · Abstract (English)
While incorporating LLMs into systems offers significant benefits in critical application areas such as healthcare, new security challenges emerge due to the potential cyber kill chain cycles that combine adversarial model, prompt injection and conventional cyber attacks. Threat modeling methods enable the system designers to identify potential cyber threats and the relevant mitigations during the early stages of development. Although the cyber security community has extensive experience in applying these methods to software-based systems, the elicited threats are usually abstract and vague, limiting their effectiveness for conducting proper likelihood and impact assessments for risk prioritization, especially in complex systems with novel attacks surfaces, such as those involving LLMs. In this study, we propose a structured, goal driven risk assessment approach that contextualizes the threats with detailed attack vectors, preconditions, and attack paths through the use of attack trees. We demonstrate the proposed approach on a case study with an LLM agent-based healthcare system. This study harmonizes the state-of-the-art attacks to LLMs with conventional ones and presents possible attack paths applicable to similar systems. By providing a structured risk assessment, this study makes a significant contribution to the literature and advances the secure-by-design practices in LLM-based systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。