保护用户提示隐私,让云端大模型推理更安全。
Confidential Prompting: Privacy-preserving LLM Inference on Cloud
- 分拆解码流程,用安全分区技术隔离用户输入与模型计算
- 在可信虚拟机中运行,确保提示和模型参数不被泄露
- 适合处理医疗、金融等敏感数据的隐私保护场景
本文提出可信提示(Confidential Prompting)的概念:在不可信的云部署大语言模型中保护用户提示,同时维持模型保密性、输出一致性与计算效率。为此,我们构建了Petridish系统,基于可信计算,核心创新为安全分区解码(SPD)。Petridish将服务拆分为两个进程:每个用户独占的预填充进程处理提示并计算注意力分数;共享的服务进程批量处理这些分数并生成输出令牌。所有操作均在可信虚拟机(CVM)内完成,其隔离机制和网络限制有效控制信息外泄。通过可验证的远程证明与开源软件栈,系统保障用户提示与模型参数的安全。Petridish在保持完整模型功能的前提下,实现了对敏感数据(如个人记录、临床资料、财务文件)的实用隐私保护推理。
原文摘要 · Abstract (English)
This paper introduces a vision of confidential prompting: securing user prompts from an untrusted, cloud-hosted large language model (LLM) while preserving model confidentiality, output invariance, and compute efficiency. As a first step toward this vision, we present Petridish, a system built on top of confidential computing and its core contribution, a novel technology called Secure Partitioned Decoding (SPD). Petridish runs the LLM service inside a confidential virtual machine (CVM), which protects the secrets, i.e., the LLM parameters and user prompts, from adversaries outside the CVM. Importantly, it splits the LLM service for a user into two processes, using SPD: a per-user process performs prefill with the user prompts and computes attention scores during decoding; a service process, shared by all users, batches the attention scores from per-user processes and generates output tokens for all users. Both the LLM provider and the users trust Petridish's CVM and its operating system, which guarantees isolation between processes and limits their outbound network capabilities to control information flow. The CVM's attestation capability and its open-source software stack enable Petridish to provide auditable protection of both user prompt and LLM confidentiality. Together, Petridish maintains full utility of LLM service and enables practical, privacy-preserving cloud-hosted LLM inference for sensitive applications, such as processing personal data, clinical records, and financial documents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。