arXiv:2511.13108cs.CV2025-11中稿 · ICML被引 1

用梯度手术保留先验,提升CLIP检测生成图像的能力

DGS-Net: Distillation-Guided Gradient Surgery for CLIP Fine-Tuning in AI-Generated Image Detection

  • 分离有害与有益梯度方向,指导优化过程
  • 在50种生成模型上平均性能领先6.6分
  • 适合需要强泛化能力的AI图像检测场景

生成模型如GANs和扩散模型的快速发展导致大量人工智能生成图像出现,引发虚假信息、隐私泄露和数字媒体信任危机。尽管CLIP等多模态大模型具备强大的迁移表征能力,但微调时常引发灾难性遗忘,削弱预训练先验并限制跨域泛化。为此,我们提出蒸馏引导梯度手术网络(DGS-Net),在保留可迁移先验的同时抑制无关成分。通过梯度空间分解,分离优化过程中的有害与有益下降方向,将任务梯度投影至有害方向的正交补空间,并对齐从冻结的CLIP编码器中蒸馏出的有益方向,实现先验保持与无关项抑制的统一优化。在50种生成模型上的大量实验表明,该方法相比现有最优方法平均提升6.6分,展现出卓越的检测性能与跨生成技术的泛化能力。

原文摘要 · Abstract (English)

The rapid progress of generative models such as GANs and diffusion models has led to the widespread proliferation of AI-generated images, raising concerns about misinformation, privacy violations, and trust erosion in digital media. Although large-scale multimodal models like CLIP offer strong transferable representations for detecting synthetic content, fine-tuning them often induces catastrophic forgetting, which degrades pre-trained priors and limits cross-domain generalization. To address this issue, we propose the Distillation-guided Gradient Surgery Network (DGS-Net), a novel framework that preserves transferable pre-trained priors while suppressing task-irrelevant components. Specifically, we introduce a gradient-space decomposition that separates harmful and beneficial descent directions during optimization. By projecting task gradients onto the orthogonal complement of harmful directions and aligning with beneficial ones distilled from a frozen CLIP encoder, DGS-Net achieves unified optimization of prior preservation and irrelevant suppression. Extensive experiments on 50 generative models demonstrate that our method outperforms state-of-the-art approaches by an average margin of 6.6, achieving superior detection performance and generalization across diverse generation techniques.

图像检测CLIP生成内容梯度手术

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。