通过降低特征互信息,让模型无法学习特定数据。
Why Do Unlearnable Examples Work: A Novel Perspective of Mutual Information
- 从互信息角度分析,削弱干净与污染特征关联可增强不可学习性。
- 新方法MI-UE使同类特征余弦相似度最大化,显著提升不可学习效果。
- 适用于数据隐私保护,尤其适合对抗防御机制的场景。
互联网上海量自由抓取数据推动了深度学习的巨大成功,也带来了数据隐私与安全的担忧。为防止未经授权的模型非法学习数据,已有多种生成不可学习样本的方法被提出,但多依赖经验启发式策略,缺乏理论支撑。本文从互信息减少的新视角分析并改进不可学习样本:有效不可学习样本始终降低干净特征与污染特征间的互信息;网络越深,互信息越低,不可学习性越强。进一步从协方差缩减角度证明,最小化类内污染特征的条件协方差能降低分布间的互信息。基于此,提出新型不可学习方法MI-UE,通过最大化类内特征余弦相似度来减小协方差,从而有效阻碍泛化。大量实验表明,该方法显著优于先前方法,即使在防御机制下仍表现优异。
原文摘要 · Abstract (English)
The volume of freely scraped data on the Internet has driven the tremendous success of deep learning. Along with this comes the growing concern about data privacy and security. Numerous methods for generating unlearnable examples have been proposed to prevent data from being illicitly learned by unauthorized deep models by impeding generalization. However, the existing approaches primarily rely on empirical heuristics, making it challenging to enhance unlearnable examples with solid explanations. In this paper, we analyze and improve unlearnable examples from a novel perspective: mutual information reduction. We demonstrate that effective unlearnable examples always decrease mutual information between clean features and poisoned features, and when the network gets deeper, the unlearnability goes better together with lower mutual information. Further, we prove from a covariance reduction perspective that minimizing the conditional covariance of intra-class poisoned features reduces the mutual information between distributions. Based on the theoretical results, we propose a novel unlearnable method called Mutual Information Unlearnable Examples (MI-UE) that reduces covariance by maximizing the cosine similarity among intra-class features, thus impeding the generalization effectively. Extensive experiments demonstrate that our approach significantly outperforms the previous methods, even under defense mechanisms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。