模型自动生成新类别数据,避免遗忘并提升未来学习能力。
Dream2Learn: Structured Generative Dreaming for Continual Learning
- 用冻结扩散模型生成新概念数据,基于自身知识进行软提示优化。
- 在多个数据集上实现正向迁移,准确率优于主流回放方法。
- 适合需要持续学习与泛化能力的智能系统,如机器人和推荐引擎。
持续学习需平衡可塑性与稳定性,避免灾难性遗忘。受人类梦境作为内部模拟与知识重构机制的启发,我们提出Dream2Learn(D2L)框架:模型自主从自身内部表示生成结构化合成经验,并用于自我优化。不同于传统生成回放重建过往数据,D2L使分类器生成语义新颖、与已有知识一致但未在训练中出现过的虚构类别。这些样本通过冻结扩散模型,以分类器自身驱动的软提示优化进行条件生成。生成数据不用于替代记忆,而是扩展并重构表征空间,使网络能对内部合成概念进行自我训练。将虚构类别融入持续训练过程,可主动构建支持未来任务迁移的潜在特征结构。该前瞻式自训练机制类比睡眠对记忆的巩固与重组,将内部模拟转化为增强泛化的能力。在Mini-ImageNet、FG-ImageNet和ImageNet-R上的实验表明,D2L持续超越强基线回放方法,并实现正向迁移,验证其通过内部生成信号提升适应性的能力。
原文摘要 · Abstract (English)
Continual learning requires balancing plasticity and stability while mitigating catastrophic forgetting. Inspired by human dreaming as a mechanism for internal simulation and knowledge restructuring, we introduce Dream2Learn (D2L), a framework in which a model autonomously generates structured synthetic experiences from its own internal representations and uses them for self-improvement. Rather than reconstructing past data as in generative replay, D2L enables a classifier to create novel, semantically distinct dreamed classes that are coherent with its learned knowledge yet do not correspond to previously observed data. These dreamed samples are produced by conditioning a frozen diffusion model through soft prompt optimization driven by the classifier itself. The generated data are not used to replace memory, but to expand and reorganize the representation space, effectively allowing the network to self-train on internally synthesized concepts. By integrating dreamed classes into continual training, D2L proactively structures latent features to support forward knowledge transfer and adaptation to future tasks. This prospective self-training mechanism mirrors the role of sleep in consolidating and reorganizing memory, turning internal simulations into a tool for improved generalization. Experiments on Mini-ImageNet, FG-ImageNet, and ImageNet-R demonstrate that D2L consistently outperforms strong rehearsal-based baselines and achieves positive forward transfer, confirming its ability to enhance adaptability through internally generated training signals.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。