用LRP技术实现无记忆重放的持续学习,兼顾解释性与隐私安全。
ReReLRP -- Remembering and Recognizing Tasks with LRP
- 利用LRP反向传播机制保留任务间关键信息,无需存储原始数据。
- 在多个数据集上达到与主流重放方法相当的性能表现。
- 适合注重模型可解释性与隐私保护的研究者使用。
深度神经网络已彻底改变众多研究领域与应用。尽管取得广泛成功,其面临一个根本性局限——灾难性遗忘:在学习新任务后无法保持对旧任务的性能。这一问题在持续学习场景中尤为严重,模型需融合新领域知识而不丢失已有能力。传统缓解方法依赖记忆回放机制,存储原始数据、原型或激活模式。虽有效,但常带来显著计算开销、隐私风险,并要求专用架构。本文提出ReReLRP(基于LRP的记忆与识别),利用层间相关性传播(LRP)保存跨任务信息。该方法在提升隐私性的同时,提供内置可解释性、灵活的模型架构与部署方式,并实现更高的内存存储效率。我们在多种数据集上验证该方法,在部分场景下性能媲美知名重放方法。
原文摘要 · Abstract (English)
Deep neural networks have revolutionized numerous research fields and applications. Despite their widespread success, a fundamental limitation known as catastrophic forgetting remains, where models fail to retain their ability to perform previously learned tasks after being trained on new ones. This limitation is particularly acute in certain continual learning scenarios, where models must integrate the knowledge from new domains with their existing capabilities. Traditional approaches to mitigate this problem typically rely on memory replay mechanisms, storing either original data samples, prototypes, or activation patterns. Although effective, these methods often introduce significant computational overhead, raise privacy concerns, and require the use of dedicated architectures. In this work we present ReReLRP (Remembering and Recognizing with LRP), a novel solution that leverages Layerwise Relevance Propagation (LRP) to preserve information across tasks. Our contribution provides increased privacy of existing replay-free methods while additionally offering built-in explainability, flexibility of model architecture and deployment, and a new mechanism to increase memory storage efficiency. We validate our approach on a wide variety of datasets, demonstrating results comparable with a well-known replay-based method in selected scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。