破解了Transformer安全推理中的混淆防御，可低成本恢复模型权重。
On the (In-)Security of the Shuffling Defense in the Transformer Secure Inference
- 通过对随机打乱的中间激活进行对齐攻击
- 在Pythia-70m和GPT-2上实现10⁻⁹~10⁻⁶的对齐误差
- 仅需约1美元查询成本即可恢复接近真实权重的模型
对于Transformer模型，密码学安全推理确保客户端仅获得最终输出，服务器无法获知客户端输入。然而，安全计算非线性层仍是主要效率瓶颈，因需大量通信轮次和数据传输。先前工作将中间激活暴露给客户端，使非线性操作可在明文下计算，虽大幅提升效率，但导致模型权重易被窃取。为缓解风险，现有方法采用混淆防御，仅向客户端披露随机打乱的激活值。本文揭示该防御并不如预期安全：我们提出一种攻击，能将不同打乱的激活对齐至共同排列，并据此提取模型权重。在Pythia-70m和GPT-2上的实验表明，所提攻击对齐误差为10⁻⁹至10⁻⁶；以约1美元查询成本，可恢复与原始权重相差10⁻⁴至10⁻²的模型权重。
原文摘要 · Abstract (English)
For Transformer models, cryptographically secure inference ensures that the client learns only the final output, while the server learns nothing about the client's input. However, securely computing nonlinear layers remains a major efficiency bottleneck due to the substantial communication rounds and data transmission required. To address this issue, prior works reveal intermediate activations to the client, allowing nonlinear operations to be computed in plaintext. Although this approach significantly improves efficiency, exposing activations enables adversaries to extract model weights. To mitigate this risk, existing works employ a shuffling defense that reveals only randomly permuted activations to the client. In this work, we show that the shuffling defense is not as robust as previously claimed. We propose an attack that aligns differently shuffled activations to a common permutation and subsequently exploits them to extract model weights. Experiments on Pythia-70m and GPT-2 demonstrate that the proposed attack can align shuffled activations with mean squared errors ranging from $10^{-9}$ to $10^{-6}$. With a query cost of approximately \$1, the adversary can recover model weights with L1-norm differences ranging from $10^{-4}$ to $10^{-2}$ compared to the oracle weights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。