arXiv:2504.08149cs.CV2025-04被引 2

用轻量方法持续学习识别新生成图像模型,不重训也能准

LoRAX: LoRA eXpandable Networks for Continual Synthetic Image Attribution

  • 基于低秩适配,为每类任务训练极小参数特征提取器
  • 在连续检测任务中表现优于或媲美顶尖方法,参数量不足3%
  • 适合需要长期更新、资源受限的图像真伪验证场景

随着生成式AI图像技术日益普及和先进,强溯源模型的需求愈发迫切。这类模型对验证图像真实性、识别其生成架构至关重要,是维护媒体完整性的关键。然而,现有溯源模型难以泛化到未见过的生成模型,传统微调方法在真实场景下也难以为继。为此,我们提出LoRA eXpandable Networks(LoRAX),一种参数高效的类别增量学习算法,可在无需全量重训练的前提下适应新型生成图像模型。该方法通过低秩适配(LoRA)为每个持续学习任务训练极参数效率的特征提取器,各任务特征提取器学习独特特征,仅需骨干模型极小部分参数。大量实验表明,LoRAX在持续深度伪造检测基准上,于所有训练场景与记忆设置下均优于或媲美当前最优算法,且每特征提取器可训练参数少于全秩实现的3%。代码已开源:https://github.com/mit-ll/lorax_cil。

原文摘要 · Abstract (English)

As generative AI image technologies become more widespread and advanced, there is a growing need for strong attribution models. These models are crucial for verifying the authenticity of images and identifying the architecture of their originating generative models-key to maintaining media integrity. However, attribution models struggle to generalize to unseen models, and traditional fine-tuning methods for updating these models have shown to be impractical in real-world settings. To address these challenges, we propose LoRA eXpandable Networks (LoRAX), a parameter-efficient class incremental algorithm that adapts to novel generative image models without the need for full retraining. Our approach trains an extremely parameter-efficient feature extractor per continual learning task via Low Rank Adaptation. Each task-specific feature extractor learns distinct features while only requiring a small fraction of the parameters present in the underlying feature extractor's backbone model. Our extensive experimentation shows LoRAX outperforms or remains competitive with state-of-the-art class incremental learning algorithms on the Continual Deepfake Detection benchmark across all training scenarios and memory settings, while requiring less than 3% of the number of trainable parameters per feature extractor compared to the full-rank implementation. LoRAX code is available at: https://github.com/mit-ll/lorax_cil.

图像溯源持续学习LoRA生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。