提出CMIF框架，兼顾大模型推理隐私与效率。
Towards Confidential and Efficient LLM Inference with Dual Privacy Protection
- 将嵌入层放于客户端可信环境，其余层部署在GPU服务器
- 优化报告噪声最大机制，仅轻微降低模型性能
- 适用于需保护用户隐私的高效大模型推理场景
基于CPU的可信执行环境（TEEs）和差分隐私（DP）已广泛用于隐私推理。由于TEEs中推理延迟高，研究者采用分片方法将线性模型组件卸载至GPU。然而，大语言模型（LLMs）中的密集非线性层导致TEEs与GPU间通信开销显著。基于DP的方法通过添加随机噪声保护数据隐私，但会损害模型性能与语义理解能力。为此，本文提出保密且高效的模型推理框架CMIF：将嵌入层安全部署于客户端的可信执行环境，后续层置于GPU服务器；同时优化报告噪声最大机制，在用户敏感输入保护上实现轻微性能下降。在Llama系列模型上的大量实验表明，CMIF有效降低了TEEs中的额外推理开销，同时保障了用户数据隐私。
原文摘要 · Abstract (English)
CPU-based trusted execution environments (TEEs) and differential privacy (DP) have gained wide applications for private inference. Due to high inference latency in TEEs, researchers use partition-based approaches that offload linear model components to GPUs. However, dense nonlinear layers of large language models (LLMs) result in significant communication overhead between TEEs and GPUs. DP-based approaches apply random noise to protect data privacy, but this compromises LLM performance and semantic understanding. To overcome the above drawbacks, this paper proposes CMIF, a Confidential and efficient Model Inference Framework. CMIF confidentially deploys the embedding layer in the client-side TEE and subsequent layers on GPU servers. Meanwhile, it optimizes the Report-Noisy-Max mechanism to protect sensitive inputs with a slight decrease in model performance. Extensive experiments on Llama-series models demonstrate that CMIF reduces additional inference overhead in TEEs while preserving user data privacy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。