普通浏览器就能偷看医疗AI聊天机器人后台，隐私漏洞令人震惊。
When RAG Chatbots Expose Their Backend: An Anonymized Case Study of Privacy and Security Risks in Patient-Facing Medical AI
- 用大模型辅助探测，发现客户端可直接获取系统配置和1000条患者对话
- 无需登录即可查看全部病历相关对话和后端接口信息
- 适合医疗AI开发者、安全审计员和政策制定者参考
基于检索增强生成（RAG）的患者面向医疗聊天机器人正被广泛推广，以提供可访问且有依据的健康信息。尽管人工智能辅助开发降低了门槛，但其仍需严格的隐私、安全与治理控制。本研究对一个公开可访问的患者面向医疗RAG聊天机器人进行了非破坏性安全评估，旨在总结生成式AI在医疗领域安全部署的治理经验。采用两阶段策略：首先利用Claude Opus 4.6进行探索性提示测试并提出漏洞假设；其次通过Chrome开发者工具手动验证，检查浏览器可见的网络流量、请求载荷、API模式、配置对象及存储的交互数据。结果显示，大模型辅助阶段发现关键漏洞：敏感系统与RAG配置通过客户端-服务器通信暴露，而非仅限于服务端。人工验证确认，普通浏览器检查即可获取系统提示词、模型与嵌入配置、检索参数、后端接口、API模式、文档与片段元数据、知识库内容以及最近1,000条患者-聊天机器人对话。该部署还违背其隐私承诺：未认证即可获取完整对话记录，包括健康相关查询。结论表明，患者面向RAG聊天机器人的严重隐私与安全缺陷，可使用标准浏览器工具识别，无需专业技能或认证；独立审查应成为部署前的必要条件。商业大模型加速了此次评估，甚至在伪装开发者身份下完成；审计者可获得的帮助，同样可能被攻击者利用。
原文摘要 · Abstract (English)
Background: Patient-facing medical chatbots based on retrieval-augmented generation (RAG) are increasingly promoted to deliver accessible, grounded health information. AI-assisted development lowers the barrier to building them, but they still demand rigorous security, privacy, and governance controls. Objective: To report an anonymized, non-destructive security assessment of a publicly accessible patient-facing medical RAG chatbot and identify governance lessons for safe deployment of generative AI in health. Methods: We used a two-stage strategy. First, Claude Opus 4.6 supported exploratory prompt-based testing and structured vulnerability hypotheses. Second, candidate findings were manually verified using Chrome Developer Tools, inspecting browser-visible network traffic, payloads, API schemas, configuration objects, and stored interaction data. Results: The LLM-assisted phase identified a critical vulnerability: sensitive system and RAG configuration appeared exposed through client-server communication rather than restricted server-side. Manual verification confirmed that ordinary browser inspection allowed collection of the system prompt, model and embedding configuration, retrieval parameters, backend endpoints, API schema, document and chunk metadata, knowledge-base content, and the 1,000 most recent patient-chatbot conversations. The deployment also contradicted its privacy assurances: full conversation records, including health-related queries, were retrievable without authentication. Conclusions: Serious privacy and security failures in patient-facing RAG chatbots can be identified with standard browser tools, without specialist skills or authentication; independent review should be a prerequisite for deployment. Commercial LLMs accelerated this assessment, including under a false developer persona; assistance available to auditors is equally available to adversaries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。