arXiv:2505.10941cs.LG2025-05ICLR被引 2

让模型既能持续学习,又能精准删除敏感信息,兼顾记忆与隐私。

Privacy-Aware Lifelong Learning

  • 通过稀疏子网络和参数共享,在单模型中实现任务增量学习。
  • 支持精确删去特定任务知识,且不降低其他任务性能。
  • 适合需要合规隐私保护的持续学习场景,如医疗、金融领域。

持续学习算法使模型能增量获取新知识而不遗忘旧知识。与此相对,机器遗忘研究旨在按要求从预训练模型中显式删除特定知识,以符合数据隐私法规中的“被遗忘权”。在高效持续学习的同时,具备选择性遗忘敏感信息的能力,是一个目标矛盾且尚未充分解决的关键挑战。本文从同时防止灾难性遗忘、促进前向知识迁移的角度出发,结合任务增量学习,实现精确的任务遗忘并最小化内存开销。提出的隐私感知持续学习(PALL)方法,基于单一神经网络架构,优化任务特定的稀疏子网络并实现参数共享。此外,引入回放式回忆机制,确保精确遗忘而无性能下降。我们在多种图像分类架构上实证了PALL的可扩展性,提供了一种兼具持续学习与隐私感知遗忘能力的先进方案,适用于负责任的人工智能应用。

原文摘要 · Abstract (English)

Lifelong learning algorithms enable models to incrementally acquire new knowledge without forgetting previously learned information. Contrarily, the field of machine unlearning focuses on explicitly forgetting certain previous knowledge from pretrained models when requested, in order to comply with data privacy regulations on the right-to-be-forgotten. Enabling efficient lifelong learning with the capability to selectively unlearn sensitive information from models presents a critical and largely unaddressed challenge with contradicting objectives. We address this problem from the perspective of simultaneously preventing catastrophic forgetting and allowing forward knowledge transfer during task-incremental learning, while ensuring exact task unlearning and minimizing memory requirements, based on a single neural network model to be adapted. Our proposed solution, privacy-aware lifelong learning (PALL), involves optimization of task-specific sparse subnetworks with parameter sharing within a single architecture. We additionally utilize an episodic memory rehearsal mechanism to facilitate exact unlearning without performance degradations. We empirically demonstrate the scalability of PALL across various architectures in image classification, and provide a state-of-the-art solution that uniquely integrates lifelong learning and privacy-aware unlearning mechanisms for responsible AI applications.

持续学习隐私保护机器遗忘

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。