arXiv:2607.09067cs.CV2026-07

用噪声级别对比学习让扩散模型兼顾图像生成与识别任务。

Probing Diffusion Denoising Dynamics for Contrastive Representation Learning

论文配图:Probing Diffusion Denoising Dynamics for Contrastive Representation Learning
图 1 · 摘自论文原文
  • 利用不同去噪步的噪声潜变量作为同一图像的随机视角,构建对比学习目标。
  • 在ImageNet-1K上线性探测准确率达80.1%,256×256无条件生成FID为5.56。
  • 仅用LoRA微调,轻量高效,适合想复用扩散模型特征的研究者。

文本到图像的扩散模型展现出惊人的生成能力,并包含可用于判别性视觉任务的丰富中间表示。我们关注的核心问题是:如何在参数高效更新下,调整预训练扩散模型的去噪动态,以支持判别性表征学习,同时保持其生成能力?为此,我们提出了D³CL。关键观察是,不同去噪步的噪声潜变量可视为同一图像的随机视角,从而将对比学习目标与标准去噪重建损失结合。该设计提供了一种简单方法,探索生成式去噪与判别性学习的交互,无需从头训练。为保持适应轻量,我们在冻结原模型参数的前提下,对预训练Stable Diffusion主干应用LoRA更新。D³CL在实证上证明了重建损失与噪声级对比损失具有互补性:在ImageNet-1K上达到80.1%线性探测准确率,256×256无条件生成的FID为5.56。额外消融实验表明,扩散特征的有效性取决于去噪状态采样的位置与方式。这些结果确立D³CL为预训练扩散模型的参数高效适配框架,表明噪声级对比学习能有效组织去噪表征用于判别任务,同时维持生成性能。

原文摘要 · Abstract (English)

Text-to-image diffusion models exhibit unprecedented generative capability and contain rich intermediate representations that can be useful for discriminative vision tasks. Motivated by this observation, we study a focused question: how can the denoising dynamics of a pretrained diffusion model be adapted to support discriminative representation learning while preserving its generative behavior under parameter-efficient updates? We present D$^3$CL as an investigation of this question. Our key observation is that noisy latents at different diffusion timesteps can be interpreted as stochastic views of the same underlying image, enabling a contrastive objective to be coupled with the standard denoising reconstruction loss. This formulation provides a simple way to probe the interaction between generative denoising and discriminative representation learning without training from scratch. To keep the adaptation lightweight, we apply LoRA updates to a pretrained Stable Diffusion backbone while freezing the original model parameters. D$^3$CL provides strong empirical evidence that reconstruction and noise-level contrastive objectives can be complementary: on ImageNet-1K, it obtains 80.1% linear-probing accuracy and an FID of 5.56 for $256 \times 256$ unconditional generation. Additional ablations on the design space suggest that the usefulness of diffusion features depends on where and how denoising states are sampled. These results establish D$^3$CL as a parameter-efficient adaptation framework for pretrained diffusion models, showing that noise-level contrastive learning can structure denoising representations for discriminative tasks while maintaining generative performance.

扩散模型对比学习表征学习LoRA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。