arXiv:2510.26311cs.LG2025-10NeurIPS被引 4

无需真实数据即可持续学习,通过分层生成伪图像保持模型性能。

Model Inversion with Layer-Specific Modeling and Alignment for Data-Free Continual Learning

  • 分层反演初始化,大幅减少反向传播迭代次数
  • 用高斯分布与对比模型对齐特征,减少伪数据偏差
  • 适用于大模型如CLIP,适合隐私敏感场景

持续学习旨在顺序训练模型的同时保留先前任务的性能。然而,由于隐私或安全限制,存储和回放数据往往不可行,且对任意预训练模型不实用。数据无关的持续学习在无历史数据情况下更新模型。除正则化外,我们采用模型反演从训练模型中合成数据,实现无需存储样本的回放。但预测模型中的反演面临两个挑战:(1) 仅从压缩输出标签生成输入会导致合成数据与真实数据漂移,回放此类数据会侵蚀先前知识;(2) 反演计算开销大,每步需全模型反向传播。这些问题在大型预训练模型(如CLIP)中尤为突出。为提高效率,我们提出逐层模型反演(PMI),受单层优化快速收敛启发。PMI为全模型反演提供强初始值,显著减少迭代次数。为缓解特征偏移,我们通过高斯分布和对比模型建模类别特征,确保合成与真实特征对齐。结合PMI与特征建模,本方法通过语义感知投影特征生成伪图像,在多个持续学习设置中实现强有效性与兼容性。

原文摘要 · Abstract (English)

Continual learning (CL) aims to incrementally train a model on a sequence of tasks while retaining performance on prior ones. However, storing and replaying data is often infeasible due to privacy or security constraints and impractical for arbitrary pre-trained models. Data-free CL seeks to update models without access to previous data. Beyond regularization, we employ model inversion to synthesize data from the trained model, enabling replay without storing samples. Yet, model inversion in predictive models faces two challenges: (1) generating inputs solely from compressed output labels causes drift between synthetic and real data, and replaying such data can erode prior knowledge; (2) inversion is computationally expensive since each step backpropagates through the full model. These issues are amplified in large pre-trained models such as CLIP. To improve efficiency, we propose Per-layer Model Inversion (PMI), inspired by faster convergence in single-layer optimization. PMI provides strong initialization for full-model inversion, substantially reducing iterations. To mitigate feature shift, we model class-wise features via Gaussian distributions and contrastive model, ensuring alignment between synthetic and real features. Combining PMI and feature modeling, our approach enables continual learning of new classes by generating pseudo-images from semantic-aware projected features, achieving strong effectiveness and compatibility across multiple CL settings.

持续学习模型反演伪数据生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。