首个安全保护扩散模型采样的通用框架，提升隐私计算效率。
CipherDM: Secure Three-Party Inference for Diffusion Model Sampling
- 用安全多方计算实现扩散模型的隐私采样，专攻非线性激活函数的加密计算。
- 在多个模型上提速1.084到2.328倍，通信开销降低1.212到1.791倍。
- 适合需要隐私保护的图像生成任务，如医疗或金融领域应用。
扩散模型（DMs）在图像生成中达到顶尖效果，并已应用于多个领域。然而，其使用过程可能严重侵犯用户隐私，因此隐私保护成为迫切问题。直接采用传统隐私计算方案（如安全多方计算，MPC）在扩散模型中面临显著的计算与通信挑战。为此，我们提出CipherDM，首个新颖、通用且可广泛部署于多种基于扩散模型任务的安全采样框架。通过深入分析采样延迟构成，识别出耗时环节，并设计针对SoftMax、SiLU和Mish等非线性激活函数的专用安全MPC协议。在主流架构（DDPM、DDIM）上使用MNIST数据集，以及通过diffusers部署的Stable Diffusion（SD）上进行评估。相较于直接在SPU上的实现，本方法运行时间提升约1.084×～2.328×，通信成本降低约1.212×～1.791×。
原文摘要 · Abstract (English)
Diffusion Models (DMs) achieve state-of-the-art synthesis results in image generation and have been applied to various fields. However, DMs sometimes seriously violate user privacy during usage, making the protection of privacy an urgent issue. Using traditional privacy computing schemes like Secure Multi-Party Computation (MPC) directly in DMs faces significant computation and communication challenges. To address these issues, we propose CipherDM, the first novel, versatile and universal framework applying MPC technology to DMs for secure sampling, which can be widely implemented on multiple DM based tasks. We thoroughly analyze sampling latency breakdown, find time-consuming parts and design corresponding secure MPC protocols for computing nonlinear activations including SoftMax, SiLU and Mish. CipherDM is evaluated on popular architectures (DDPM, DDIM) using MNIST dataset and on SD deployed by diffusers. Compared to direct implementation on SPU, our approach improves running time by approximately 1.084\times \sim 2.328\times, and reduces communication costs by approximately 1.212\times \sim 1.791\times.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。