用轻量模块化安全中间件防止健康类AI给出有害建议
Preventing Another Tessa: Modular Safety Middleware For Health-Adjacent AI Assistants
- 结合词法过滤与大模型策略检查，单次调用完成安全判断
- 合成测试中完美拦截所有不当请求，延迟与成本不变
- 适合医疗类AI开发者及安全治理人员参考落地
2023年，美国饮食障碍协会（NEDA）的聊天机器人Tessa因向脆弱用户推荐有害减肥建议而被暂停，暴露出医疗类AI缺乏安全工程的严重风险。本文以Tessa为案例，提出一种轻量级、模块化的混合安全中间件，融合确定性词法门禁与内联大语言模型（LLM）策略过滤器，在单次模型调用中实现‘失败闭合’判断与升级路径。通过合成评估验证，该设计在零额外开销下实现对所有不安全提示的完美拦截，优于传统多阶段流水线。研究还将Tessa的失败模式映射至OWASP LLM Top10和NIST SP 800-53等框架，建立技术防护与可操作治理控制的关联。结果表明，健康相关AI的稳健与可审计安全无需重型基础设施：在最后环节部署明确、可测试的检查机制即可避免‘再出一个Tessa’，结合治理与升级机制保障实际部署可持续性。
原文摘要 · Abstract (English)
In 2023, the National Eating Disorders Association's (NEDA) chatbot Tessa was suspended after providing harmful weight-loss advice to vulnerable users-an avoidable failure that underscores the risks of unsafe AI in healthcare contexts. This paper examines Tessa as a case study in absent safety engineering and demonstrates how a lightweight, modular safeguard could have prevented the incident. We propose a hybrid safety middleware that combines deterministic lexical gates with an in-line large language model (LLM) policy filter, enforcing fail-closed verdicts and escalation pathways within a single model call. Using synthetic evaluations, we show that this design achieves perfect interception of unsafe prompts at baseline cost and latency, outperforming traditional multi-stage pipelines. Beyond technical remedies, we map Tessa's failure patterns to established frameworks (OWASP LLM Top10, NIST SP 800-53), connecting practical safeguards to actionable governance controls. The results highlight that robust, auditable safety in health-adjacent AI does not require heavyweight infrastructure: explicit, testable checks at the last mile are sufficient to prevent "another Tessa", while governance and escalation ensure sustainability in real-world deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。