arXiv:2510.02631cs.CV2025-10

用函数型低秩适配防止生成模型遗忘,仅需当前任务数据即可持续学习。

Deep Generative Continual Learning using Functional LoRA: FunLoRA

  • 提出函数型低秩适配(FunLoRA),用函数动态增强单秩矩阵表达能力。
  • 在流匹配模型上实现更高分类准确率,内存和采样时间仅为之前的几分之一。
  • 适合需要高效持续学习的生成模型应用,如文本-视觉生成系统。

深度生成模型的持续适应具有巨大潜力与重要意义,因其在文本与视觉应用中快速普及。然而,增量训练面临灾难性遗忘问题,导致神经网络难以有效融合新知识。常见策略是利用模型自身生成的合成数据进行再训练以缓解遗忘,但存在两大局限:(i) 随着持续训练,时间成本不断累积,最终不可行;(ii) 依赖合成数据导致长期性能下降,因合成样本缺乏真实数据的丰富性。本文提出一种基于低秩适配(LoRA)的新颖且更富表达力的条件机制,仅使用一秩矩阵,并通过精心选择的函数对其重构矩阵秩进行功能提升,命名为函数型低秩适配(FunLoRA)。该动态条件机制确保生成模型避免灾难性遗忘,且只需在当前任务数据上训练。大量实验基于从头训练的流匹配模型表明,所提出的参数高效微调(PEFT)方法超越基于扩散模型的现有最优结果,在获得更高分类准确率的同时,仅需极少的内存开销和采样时间。

原文摘要 · Abstract (English)

Continual adaptation of deep generative models holds tremendous potential and critical importance, given their rapid and expanding usage in text and vision based applications. Incremental training, however, remains highly challenging due to catastrophic forgetting phenomenon, which makes it difficult for neural networks to effectively incorporate new knowledge. A common strategy consists in retraining the generative model on its own synthetic data in order to mitigate forgetting. Yet, such an approach faces two major limitations: (i) the continually increasing training time eventually becomes intractable, and (ii) reliance on synthetic data inevitably leads to long-term performance degradation, since synthetic samples lack the richness of real training data. In this paper, we attenuate these issues by designing a novel and more expressive conditioning mechanism for generative models based on low rank adaptation (LoRA), that exclusively employs rank 1 matrices, whose reparametrized matrix rank is functionally increased using carefully selected functions -- and dubbed functional LoRA: FunLoRA. Using this dynamic conditioning, the generative model is guaranteed to avoid catastrophic forgetting and needs only to be trained on data from the current task. Extensive experiments using flow-matching based models trained from scratch, showcase that our proposed parameter-efficient fine-tuning (PEFT) method surpasses prior state-of-the-art results based on diffusion models, reaching higher classification accuracy scores, while only requiring a fraction of the memory cost and sampling time.

生成模型持续学习参数高效

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。