CrypTorch让多方计算机器学习编译更透明,自动优化速度且不损失精度。
CrypTorch: PyTorch-based Auto-tuning Compiler for Machine Learning with Multi-party Computation
- 将加密计算的转换步骤模块化,可逐个检查优化
- 自动选择最优转换组合,无精度损失时提速1.21-1.5倍
- 基于PyTorch 2设计,适合需要隐私保护的开发者
基于多方计算(MPC)的机器学习可在不共享私有数据或模型参数的情况下跨多方运行。然而,现有框架因一系列添加误差和开销的MPC特有转换,常导致准确率和性能下降。这些转换对用户不透明,难以定位和优化。我们提出CrypTorch,一个模块化、可扩展且支持迭代测试的编译器框架。它将MPC转换拆分为独立阶段,每步生成可执行图,便于定位问题。基于此,CrypTorch在算子近似阶段自动选择转换组合以平衡性能与准确率,该阶段是主要失真来源。仅靠自动调优即实现1.21-1.5倍速度提升且无精度损失,允许部分精度下降时达1.33-1.74倍。结合更好工程实践(因模块化设计更易采用),相比主流框架CrypTen实现3.74-8.32倍端到端加速。CrypTorch作为PyTorch 2编译器的扩展构建。
原文摘要 · Abstract (English)
MPC-based ML uses multi-party computation (MPC) to run machine learning (ML) workloads across multiple parties without each having to share their private data or model parameters. However, existing frameworks frequently degrade accuracy and performance due to a series of MPC-specific transformations that add errors and overheads. These transformations are mostly opaque to users, making it hard to find and/or optimize problematic transformations. We propose CrypTorch, a modular, extensible, and iteratively-testable compiler framework for MPC-based ML. CrypTorch splits MPC-specific transformations into modular compilation stages, allowing users to easily inspect and optimize them. CrypTorch emits an executable graph after each transformation, allowing iterative testing to pinpoint any problematic transformations. Building on these features, CrypTorch automatically chooses a set of transformations from a pool of choices to balance performance and accuracy during the operator approximation stage, which we identified as the biggest contributor to accuracy/performance degradation. CrypTorch's auto-tuning alone provides 1.21-1.5x speedup without accuracy loss, and 1.33-1.74x speedup when some accuracy degradation is allowed. Combined with better engineering and adoption of state-of-the-art practices (made easier due to CrypTorch's modular design), CrypTorch brings 3.74-8.32x end-to-end speedup compared to the popular CrypTen. CrypTorch is built as an extension to PyTorch 2's compiler.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。