首个实现全加密训练的深度神经网络框架,支持无需交互的隐私保护学习。
ReBoot: Encrypted Training of Deep Neural Networks with CKKS Bootstrapping
- 基于CKKS方案设计新架构,用局部误差信号降低计算深度和噪声积累。
- 采用实数打包策略,结合近似自举技术,实现任意深度网络的高效加密训练。
- 在图像与表格数据上精度媲美32位浮点训练,比现有方法快8.83倍。
数据隐私日益受关注,亟需能在不泄露信息的前提下处理敏感数据的深度学习方法。同态加密(HE)因其抗量子安全性和端到端保护能力脱颖而出,可在计算过程中保障数据安全。尽管深度神经网络(DNN)在HE环境中备受关注,但其应用长期局限于加密推理。此前研究多集中于逻辑回归或依赖多方计算进行微调,因加密训练存在巨大计算开销与算法复杂性而受限。本文提出ReBoot,首个支持全加密、非交互式DNN训练的框架。基于CKKS方案,ReBoot引入一种新型HE兼容神经网络架构,利用局部误差信号以最小化乘法深度并减少噪声累积;采用定制打包策略,通过SIMD操作实现实数运算,显著降低计算与内存开销;结合近似自举技术,使学习算法可有效训练任意深度的多层感知机,适用于机器学习即服务场景。在图像识别与表格基准测试中,其准确率接近32位浮点明文训练水平,较加密逻辑回归提升最高达+3.27%,较现有加密DNN框架提升最高达+6.83%,训练延迟降低最多达8.83倍。ReBoot已开源供科研社区使用。
原文摘要 · Abstract (English)
Growing concerns over data privacy underscore the need for deep learning methods capable of processing sensitive information without compromising confidentiality. Among privacy-enhancing technologies, Homomorphic Encryption (HE) stands out by providing post-quantum cryptographic security and end-to-end data protection, safeguarding data even during computation. While Deep Neural Networks (DNNs) have gained attention in HE settings, their use has largely been restricted to encrypted inference. Prior research on encrypted training has primarily focused on logistic regression or has relied on multi-party computation to enable model fine-tuning. This stems from the substantial computational overhead and algorithmic complexity involved in DNNs training under HE. In this paper, we present ReBoot, the first framework to enable fully encrypted and non-interactive training of DNNs. Built upon the CKKS scheme, ReBoot introduces a novel HE-compliant neural network architecture based on local error signals, specifically designed to minimize multiplicative depth and reduce noise accumulation. ReBoot employs a tailored packing strategy that leverages real-number arithmetic via SIMD operations, significantly lowering both computational and memory overhead. Furthermore, by integrating approximate bootstrapping, ReBoot learning algorithm supports effective training of arbitrarily deep multi-layer perceptrons, making it well-suited for machine learning as-a-service. ReBoot is evaluated on both image recognition and tabular benchmarks, achieving accuracy comparable to 32-bit floating-point plaintext training while enabling fully encrypted training. It improves test accuracy by up to +3.27% over encrypted logistic regression, and up to +6.83% over existing encrypted DNN frameworks, while reducing training latency by up to 8.83x. ReBoot is made available to the scientific community as a public repository.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。