让预训练模型学会忘记特定信息，同时保留关键知识并学习新内容。
Knowledge Swapping via Learning and Unlearning
- 先学后忘：按从低层到高层的顺序学习，再从高层向底层遗忘。
- 在图像分类、检测和分割任务上验证了方法的有效性。
- 适合需要可控知识更新的模型维护与隐私保护场景。
我们提出一种名为知识交换的新任务，旨在通过选择性地遗忘用户指定的信息，同时保留重要知识并获取新知识，实现对预训练模型知识的精细调控。通过对特征层级结构的深入分析发现，增量学习通常从低层表示逐步推进到高层语义，而遗忘则相反，从高层语义开始向下扩散至低层特征。基于此，我们采用“先学后忘”策略构建知识交换基准。在图像分类、目标检测和语义分割等多样化任务上的综合实验验证了该策略的有效性。源代码已公开于 https://github.com/xingmingyu123456/KnowledgeSwapping。
原文摘要 · Abstract (English)
We introduce \textbf{Knowledge Swapping}, a novel task designed to selectively regulate knowledge of a pretrained model by enabling the forgetting of user\-specified information, retaining essential knowledge, and acquiring new knowledge simultaneously. By delving into the analysis of knock-on feature hierarchy, we find that incremental learning typically progresses from low\-level representations to higher\-level semantics, whereas forgetting tends to occur in the opposite direction\-starting from high-level semantics and moving down to low-level features. Building upon this, we propose to benchmark the knowledge swapping task with the strategy of \textit{Learning Before Forgetting}. Comprehensive experiments on various tasks like image classification, object detection, and semantic segmentation validate the effectiveness of the proposed strategy. The source code is available at \href{https://github.com/xingmingyu123456/KnowledgeSwapping}{https://github.com/xingmingyu123456/KnowledgeSwapping}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。