arXiv:2410.08015cs.LG2024-10ECCV被引 5

通过剪枝限制大模型知识迁移,防止未经授权使用。

Non-transferable Pruning

  • 用剪枝和新损失函数降低模型在非授权数据上的可迁移性。
  • 在多对源-目标域上平均SLC-AUC达-0.54,显著优于现有方法。
  • 适合保护预训练模型知识产权,尤其防范非法迁移学习。

预训练深度神经网络(DNN)因基于海量数据整合了多方面知识,日益被视为重要知识产权(IP)。为防止模型被侵权,现有策略多聚焦于限制模型直接访问,而本文关注更广义的适用授权问题,旨在阻止模型知识在未经授权的迁移学习场景中被滥用。提出非可迁移剪枝(NTP)方法,利用剪枝控制预训练DNN在非授权数据域上的可迁移性。通过交替方向乘子法(ADMM)优化模型稀疏度与新型非可迁移学习损失,并引入费雪空间判别正则化以约束模型在目标数据集上的泛化能力。设计新指标样本学习曲线下的面积(SLC-AUC),可全面评估全微调下不同样本量的表现。实验表明,NTP在多组源-目标域上平均SLC-AUC达-0.54,显著优于当前最优方法,证明其有效抑制非法迁移。该方法在监督与自监督学习中均表现稳健,具备实际应用价值。

原文摘要 · Abstract (English)

Pretrained Deep Neural Networks (DNNs), developed from extensive datasets to integrate multifaceted knowledge, are increasingly recognized as valuable intellectual property (IP). To safeguard these models against IP infringement, strategies for ownership verification and usage authorization have emerged. Unlike most existing IP protection strategies that concentrate on restricting direct access to the model, our study addresses an extended DNN IP issue: applicability authorization, aiming to prevent the misuse of learned knowledge, particularly in unauthorized transfer learning scenarios. We propose Non-Transferable Pruning (NTP), a novel IP protection method that leverages model pruning to control a pretrained DNN's transferability to unauthorized data domains. Selective pruning can deliberately diminish a model's suitability on unauthorized domains, even with full fine-tuning. Specifically, our framework employs the alternating direction method of multipliers (ADMM) for optimizing both the model sparsity and an innovative non-transferable learning loss, augmented with Fisher space discriminative regularization, to constrain the model's generalizability to the target dataset. We also propose a novel effective metric to measure the model non-transferability: Area Under the Sample-wise Learning Curve (SLC-AUC). This metric facilitates consideration of full fine-tuning across various sample sizes. Experimental results demonstrate that NTP significantly surpasses the state-of-the-art non-transferable learning methods, with an average SLC-AUC at $-0.54$ across diverse pairs of source and target domains, indicating that models trained with NTP do not suit for transfer learning to unauthorized target domains. The efficacy of NTP is validated in both supervised and self-supervised learning contexts, confirming its applicability in real-world scenarios.

模型保护迁移学习剪枝知识产权

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。