用低秩分解加速安全多方计算中的神经网络推理。
LRD-MPC: Efficient MPC Inference through Low-rank Decomposition
- 通过低秩分解将大矩阵乘法拆成两个小乘法，降低计算开销。
- 实验显示在3方计算中提速33%，GPU能耗降低52%。
- 适合需要隐私保护的云上模型推理场景。
安全多方计算（MPC）允许多个互不信任的参与方在不泄露输入的前提下联合计算函数。其在机器学习中的应用尤其受到关注，特别是在跨多个云虚拟机（VM）部署的安全推理服务中，每个VM作为一方。模型提供方对模型权重进行秘密分享，用户对输入进行秘密分享，确保每个服务器仅处理随机分片。尽管MPC提供强加密保障，但其带来显著的计算和通信开销。深度神经网络依赖大量卷积层和全连接层，这些层在MPC中需执行代价高昂的矩阵乘法。为降低此开销，我们提出对线性层采用低秩分解（LRD），将一个大型矩阵乘法替换为两个较小的乘法。然而，每轮矩阵乘法在MPC中对应一次通信回合，分解导致额外通信回合；此外，新增乘法还需额外截断以维持数值精度，而截断本身也需通信与计算，可能抵消分解带来的收益。为此，我们引入两项互补优化：截断跳过和线性层高效拼接。截断跳过消除因LRD引入的额外截断，线性层拼接则通过操作流水线隐藏额外通信回合。两者共同缓解了LRD在MPC中的主要开销，提升整体效率。该方法适用于各类MPC协议。实验表明，在n-PC中最高提速25%，在3-PC中最高提速33%，同时实现高达52%的GPU能耗节省和88%的离线阶段延迟减少。
原文摘要 · Abstract (English)
Secure Multi-party Computation (MPC) enables untrusted parties to jointly compute a function without revealing their inputs. Its application to machine learning (ML) has gained significant attention, particularly for secure inference services deployed across multiple cloud virtual machines (VMs), where each VM acts as an MPC party. Model providers secret-share model weights, and users secret-share inputs, ensuring that each server operates only on random shares. While MPC provides strong cryptographic guarantees, it incurs substantial computational and communication overhead. Deep neural networks rely heavily on convolutional and fully connected layers, which require costly matrix multiplications in MPC. To reduce this cost, we propose leveraging low-rank decomposition (LRD) for linear layers, replacing one large matrix multiplication with two smaller ones. Each matrix multiplication in MPC incurs a round of communication, meaning decomposing one matrix multiplication into two leads to an additional communication round. Second, the added matrix multiplication requires an additional truncation step to maintain numerical precision. Since truncation itself requires communication and computation, these overheads can offset the gains from decomposition. To address this, we introduce two complementary optimizations: truncation skipping and efficient linear layer concatenation. Truncation skipping removes the extra truncation induced by LRD, while linear layer concatenation pipelines operations to hide the additional communication round. Together, these techniques mitigate the main overheads of LRD in MPC and improve overall efficiency. Our approach is broadly applicable across MPC protocols. Experiments show up to 25% speedup in n-PC and 33% in 3-PC protocols over full-rank baselines, along with up to 52% GPU energy savings and 88% reduction in offline-phase latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。