通过优化加密计算流程,实现高效安全训练,速度提升5.3倍。
CryptoTrain: Fast Secure Training on Encrypted Dataset
- 混合使用同态加密与不经意传输,分别处理线性和非线性运算。
- 预计算密文乘法并优化多项式卷积,减少90%以上计算开销。
- 适合需要隐私保护的金融、医疗等敏感数据训练场景。
安全训练在保护数据和模型权重机密性的同时,通常带来显著的训练开销。传统的全同态加密(FHE)非交互式训练模型因计算量大的置位操作而负担沉重。为此,我们提出CryptoTrain-B框架,结合FHE与不经意传输(OT),分别用于处理线性与非线性操作,消除昂贵的置位操作需求。尽管如此,密文-密文乘法(CCMul)仍是加密输入与模型交互中的关键瓶颈。本文提出CCMul-Precompute技术,将CCMul离线预计算,训练时改用更轻量的密文-明文乘法(CPMul)。此外,传统FHE系统中多项式卷积会引入无关冗余值,需额外多项式与密文表示输入,导致额外乘法。为此,我们引入相关多项式卷积,仅将相关输入值编码至多项式,大幅减少计算量与开销。将两项优化整合至CryptoTrain-B后,形成高效安全训练框架CryptoTrain。大量实验表明,相比先前方法,CryptoTrain实现约5.3倍的训练时间加速。
原文摘要 · Abstract (English)
Secure training, while protecting the confidentiality of both data and model weights, typically incurs significant training overhead. Traditional Fully Homomorphic Encryption (FHE)-based non-inter-active training models are heavily burdened by computationally demanding bootstrapping. To develop an efficient secure training system, we established a foundational framework, CryptoTrain-B, utilizing a hybrid cryptographic protocol that merges FHE with Oblivious Transfer (OT) for handling linear and non-linear operations, respectively. This integration eliminates the need for costly bootstrapping. Although CryptoTrain-B sets a new baseline in performance, reducing its training overhead remains essential. We found that ciphertext-ciphertext multiplication (CCMul) is a critical bottleneck in operations involving encrypted inputs and models. Our solution, the CCMul-Precompute technique, involves precomputing CCMul offline and resorting to the less resource-intensive ciphertext-plaintext multiplication (CPMul) during private training. Furthermore, conventional polynomial convolution in FHE systems tends to encode irrelevant and redundant values into polynomial slots, necessitating additional polynomials and ciphertexts for input representation and leading to extra multiplications. Addressing this, we introduce correlated polynomial convolution, which encodes only related input values into polynomials, thus drastically reducing the number of computations and overheads. By integrating CCMul-Precompute and correlated polynomial convolution into CryptoTrain-B, we facilitate a rapid and efficient secure training framework, CryptoTrain. Extensive experiments demonstrate that CryptoTrain achieves a ~5.3X training time reduction compared to prior methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。