动态调整剪裁阈值,提升隐私保护下的小样本学习泛化能力
Adaptive Clipping for Privacy-Preserving Few-Shot Learning: Enhancing Generalization with Limited Data
- 自适应剪裁动态调节训练中的梯度剪裁阈值
- 在多个基准数据集上显著降低隐私保护导致的性能下降
- 适合需要高隐私性且数据稀缺的现实场景
在数据驱动的机器学习应用中,隐私问题与标注数据稀缺已成为关键挑战,尤其在小样本学习领域尤为突出。为应对这一挑战,隐私保护的小样本学习方法应运而生。然而,现有隐私保护技术常因隐私与性能之间的根本权衡而导致模型效用下降。为此,本文提出一种名为Meta-Clip的新方法,专为元学习算法(包括差分隐私的MAML、DP-Reptile和DP-MetaSGD)设计,通过动态调整剪裁阈值,在保障数据隐私的同时最大化学习能力。该方法实现了对敏感信息泄露的细粒度控制,缓解了小数据集上的过拟合问题,显著提升了元学习模型的泛化性能。在多个基准数据集上的全面实验表明,该方法有效降低了效用损失,相较现有技术展现出更优的隐私-效用权衡。Adaptive Clipping的引入为隐私保护的小样本学习领域带来重要进展,有助于构建适用于数据有限场景的安全可靠模型。
原文摘要 · Abstract (English)
In the era of data-driven machine-learning applications, privacy concerns and the scarcity of labeled data have become paramount challenges. These challenges are particularly pronounced in the domain of few-shot learning, where the ability to learn from limited labeled data is crucial. Privacy-preserving few-shot learning algorithms have emerged as a promising solution to address such pronounced challenges. However, it is well-known that privacy-preserving techniques often lead to a drop in utility due to the fundamental trade-off between data privacy and model performance. To enhance the utility of privacy-preserving few-shot learning methods, we introduce a novel approach called Meta-Clip. This technique is specifically designed for meta-learning algorithms, including Differentially Private (DP) model-agnostic meta-learning, DP-Reptile, and DP-MetaSGD algorithms, with the objective of balancing data privacy preservation with learning capacity maximization. By dynamically adjusting clipping thresholds during the training process, our Adaptive Clipping method provides fine-grained control over the disclosure of sensitive information, mitigating overfitting on small datasets and significantly improving the generalization performance of meta-learning models. Through comprehensive experiments on diverse benchmark datasets, we demonstrate the effectiveness of our approach in minimizing utility degradation, showcasing a superior privacy-utility trade-off compared to existing privacy-preserving techniques. The adoption of Adaptive Clipping represents a substantial step forward in the field of privacy-preserving few-shot learning, empowering the development of secure and accurate models for real-world applications, especially in scenarios where there are limited data availability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。