用引导扩散模型从中间表示重建图像,暴露大模型分片推理隐私风险
DRAG: Data Reconstruction Attack using Guided Diffusion
- 基于预训练扩散模型的先验知识,迭代重构中间特征中的原始图像
- 在视觉基础模型深层中间表示上实现高保真图像重建,效果超越现有方法
- 揭示分片推理中大模型隐私漏洞,适合关注模型安全的研究者
随着大模型兴起,分片推理(SI)成为部署模型于轻量边缘设备与云端服务器的流行计算范式,缓解了数据隐私与计算成本问题。然而,现有数据重建攻击多针对小型卷积神经网络分类模型,对基础模型在SI场景下的隐私风险研究不足。为此,我们提出一种基于引导扩散的新数据重建攻击方法,利用在大规模数据集上预训练的潜在扩散模型(LDM)所蕴含的丰富先验知识。该方法在LDM学习到的图像先验基础上进行迭代重建,能有效从中间表示(IR)生成与原始数据高度相似的高质量图像。大量实验表明,该方法在视觉基础模型深层中间表示的数据重建任务上,无论定性还是定量均显著优于现有最佳方法,凸显了在分片推理场景中为大模型设计更鲁棒隐私保护机制的紧迫性。代码已开源:https://github.com/ntuaislab/DRAG。
原文摘要 · Abstract (English)
With the rise of large foundation models, split inference (SI) has emerged as a popular computational paradigm for deploying models across lightweight edge devices and cloud servers, addressing data privacy and computational cost concerns. However, most existing data reconstruction attacks have focused on smaller CNN classification models, leaving the privacy risks of foundation models in SI settings largely unexplored. To address this gap, we propose a novel data reconstruction attack based on guided diffusion, which leverages the rich prior knowledge embedded in a latent diffusion model (LDM) pre-trained on a large-scale dataset. Our method performs iterative reconstruction on the LDM's learned image prior, effectively generating high-fidelity images resembling the original data from their intermediate representations (IR). Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods, both qualitatively and quantitatively, in reconstructing data from deep-layer IRs of the vision foundation model. The results highlight the urgent need for more robust privacy protection mechanisms for large models in SI scenarios. Code is available at: https://github.com/ntuaislab/DRAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。