提出首个多项式时间端到端模型提取方法,可攻破更深神经网络。
Navigating the Deep: End-to-End Extraction on Deep Neural Networks
- 重构签名提取流程,解决深度网络中的秩不足与噪声传播问题。
- 在MNIST/CIFAR-10上实现至少8层网络的稳定提取,远超此前3层上限。
- 结合多项式方法避免指数级搜索,适合研究模型安全的开发者。
神经网络模型提取已成为重要安全威胁,攻击者可通过黑盒查询恢复模型参数。Carlini等人在CRYPTO'20提出两阶段方法:签名提取与符号提取,但前者仅适用于极浅网络,后者为指数时间复杂度。近期Canales-Martinez等人(Eurocrypt'24)提出多项式时间符号提取法,但依赖已成功提取的签名,且对低置信度神经元失效。本文首次系统分析并修正Carlini方法中签名提取的关键缺陷,包括秩不足与深层噪声传播。针对此提出高效算法解决方案,显著提升网络深度支持能力。同时改进签名提取数值精度,并结合两种多项式方法增强符号提取,避免低置信度神经元下的指数搜索。最终实现首个完全多项式时间的端到端模型提取方法。在基于ReLU的神经网络上通过大量实验验证,显著提升提取深度:在MNIST或CIFAR-10训练的网络中,能一致提取至少八层,而先前方法仅能提取类似宽度网络的前三层。
原文摘要 · Abstract (English)
Neural network model extraction has recently emerged as an important security concern, as adversaries attempt to recover a network's parameters via black-box queries. Carlini et al. proposed in CRYPTO'20 a model extraction approach, consisting of two steps: signature extraction and sign extraction. However, in practice this signature-extraction method is limited to very shallow networks only, and the proposed sign-extraction method is exponential in time. Recently, Canales-Martinez et al. (Eurocrypt'24) proposed a polynomial-time sign-extraction method, but it assumes the corresponding signatures have already been successfully extracted and can fail on so-called low-confidence neurons. In this work, we first revisit and refine the signature extraction process by systematically identifying and addressing for the first time critical limitations of Carlini et al.'s signature-extraction method. These limitations include rank deficiency and noise propagation from deeper layers. To overcome these challenges, we propose efficient algorithmic solutions for each of the identified issues. Our approach permits the extraction of much deeper networks than previously possible. In addition, we propose new methods to improve numerical precision in signature extraction, and enhance the sign extraction part by combining two polynomial methods to avoid exponential exhaustive search in the case of low-confidence neurons. This leads to the very first end-to-end model extraction method that runs in polynomial time. We validate our attack through extensive experiments on ReLU-based neural networks, demonstrating significant improvements in extraction depth. For instance, our attack extracts consistently at least eight layers of neural networks trained on either the MNIST or CIFAR-10 datasets, while previous works could barely extract the first three layers of networks of similar width.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。