用TRiSM框架让医疗AI代理更安全,攻击成功率降三成,报告准确率升14个百分点。
Why Trust Your Agent? Empirical Security Gains from TRiSM-Guided Agentic Workflows in Healthcare

- 通过TRiSM框架设计最小权限、纵深防御的AI代理流程
- 对抗攻击成功率从31%降至10%,数据注入攻击下降至25%
- 适合关注AI医疗安全与合规的开发者和系统设计者
基于AI代理的系统通过将应用工具暴露给大语言模型(LLMs)实现任务自动化,但为提升能力常赋予代理超出普通用户的权限,带来严重安全隐患。本文将人工智能可信、风险与安全管理(TRiSM)框架应用于医疗报告生成系统,验证了如何将不安全的代理工作流转化为安全导向的流程。在五种LLM(Claude Haiku 4.5、GPT-4.1-nano、GPT-4.1-mini、GPT-5.4-mini、Gemini 2.5 Flash)上,针对两类报告共生成800份报告,并测试500种攻击场景,包括RAG污染、数据字段注入和客户端网络注入。经TRiSM指导的工作流使RAG污染攻击成功率由31%降至10%,数据字段注入由42%降至25%,并通过服务端提示构建彻底消除网络注入风险。同时报告准确率提升14个百分点(72.5%→86.5%),证明安全设计可同时提升可靠性。研究强调最小权限与纵深防御对提升安全性和准确性的重要性,也指出模型选择是架构设计的关键考量。
原文摘要 · Abstract (English)
Agent-based AI has enabled the automation of tasks by exposing application tools and resources to large language models (LLMs). However, to improve scope and accuracy, agents are often given access rights that exceed those of ordinary users, introducing significant security risks. AI is routinely integrated into applications with a disregard to security, risking data exposure and breaching regulations. This paper applies the AI Trust, Risk, and Security Management (TRiSM) framework to a medical report-generation application to demonstrate how an insecure agent workflow can be transformed into security-conscious agentic workflow. Both workflows were evaluated across five LLMs (Claude Haiku 4.5, GPT-4.1-nano, GPT-4.1-mini, GPT-5.4-mini, and Gemini 2.5 Flash) on two report types, totalling 800 generations and 500 attack scenarios including RAG poisoning, data-field injection, and client-side network injection. The TRiSM-guided agentic workflow reduced mean attack success rates from 31% to 10% for RAG poisoning and from 42% to 25% for data-field injection, while eliminating the network injection vector entirely through server-side prompt construction. Furthermore, report accuracy increased by 14 percentage points (72.5% to 86.5%) with the agentic workflow, demonstrating a secure design which provides more reliable outputs. This paper contributes to knowledge by demonstrating least-privilege, defence in depth agentic workflows improving security and accuracy, while also highlighting model choice is a necessary architectural consideration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。