arXiv:2503.00383cs.LGcs.AI2025-03CVPR被引 10

提出条件熵最大化方法,显著提升协作推理中的隐私保护能力。

Theoretical Insights in Model Inversion Robustness and Conditional Entropy Maximization for Collaborative Inference Systems

  • 通过最大化输入条件熵来理论约束数据重建误差下限。
  • 在4个数据集上平均提升攻击防御效果12.9%至48.2%。
  • 可无缝集成到现有防护方法中,不牺牲性能与效率。

通过将原始数据本地编码为中间特征,协作推理使用户能在不向云端暴露敏感原始数据的情况下使用强大深度学习模型。然而,近期研究发现这些中间特征可能仍存在隐私泄露风险,攻击者可通过模型逆向攻击(MIA)重构原始数据。基于扰动的方法(如噪声注入、对抗表征学习、信息过滤)虽能经验性增强逆向鲁棒性,但缺乏对冗余信息量化的手段,且未建立冗余最小化与逆向鲁棒性提升之间的明确数学关系。本文首次理论证明:给定中间特征时输入的条件熵,是任何MIA下重建均方误差(MSE)的严格下界。进而基于高斯混合估计,提出可微分且可解的条件熵上界逼近方法,并设计条件熵最大化(CEM)算法以增强逆向鲁棒性。在四个数据集上的实验表明,所提CEM无需牺牲特征效用与计算效率,即可稳定提升各类基于扰动的防御机制的鲁棒性,平均增益达12.9%至48.2%。代码已开源。

原文摘要 · Abstract (English)

By locally encoding raw data into intermediate features, collaborative inference enables end users to leverage powerful deep learning models without exposure of sensitive raw data to cloud servers. However, recent studies have revealed that these intermediate features may not sufficiently preserve privacy, as information can be leaked and raw data can be reconstructed via model inversion attacks (MIAs). Obfuscation-based methods, such as noise corruption, adversarial representation learning, and information filters, enhance the inversion robustness by obfuscating the task-irrelevant redundancy empirically. However, methods for quantifying such redundancy remain elusive, and the explicit mathematical relation between this redundancy minimization and inversion robustness enhancement has not yet been established. To address that, this work first theoretically proves that the conditional entropy of inputs given intermediate features provides a guaranteed lower bound on the reconstruction mean square error (MSE) under any MIA. Then, we derive a differentiable and solvable measure for bounding this conditional entropy based on the Gaussian mixture estimation and propose a conditional entropy maximization (CEM) algorithm to enhance the inversion robustness. Experimental results on four datasets demonstrate the effectiveness and adaptability of our proposed CEM; without compromising feature utility and computing efficiency, plugging the proposed CEM into obfuscation-based defense mechanisms consistently boosts their inversion robustness, achieving average gains ranging from 12.9\% to 48.2\%. Code is available at \href{https://github.com/xiasong0501/CEM}{https://github.com/xiasong0501/CEM}.

隐私保护模型逆向条件熵协作推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。