用原型学习替代外部记忆，实现高效无存储的图像生成。
Prototype-Guided Diffusion: Visual Conditioning without External Memory
- 在扩散过程中嵌入对比学习，自动构建紧凑视觉原型。
- 生成质量接近传统方法，计算与存储成本显著降低。
- 适合需要轻量级、可扩展图像生成的场景。
扩散模型虽能生成高质量图像，但因迭代去噪而计算开销大。潜空间模型如Stable Diffusion虽降低负担，却损失细节；检索增强方法提升效率，却依赖大型内存库、静态相似度模型和固定架构。本文提出原型扩散模型（PDM），将原型学习融入扩散过程，实现无需外部存储的自适应条件生成。PDM通过对比学习从干净特征中学习紧凑视觉原型，并在去噪过程中将噪声表示与语义相关模式对齐。实验表明，PDM在保持高生成质量的同时，显著降低计算与存储开销，为检索式条件生成提供可扩展的新方案。
原文摘要 · Abstract (English)
Diffusion models achieve state-of-the-art image generation but remain computationally costly due to iterative denoising. Latent-space models like Stable Diffusion reduce overhead yet lose fine detail, while retrieval-augmented methods improve efficiency but rely on large memory banks, static similarity models, and rigid infrastructures. We introduce the Prototype Diffusion Model (PDM), which embeds prototype learning into the diffusion process to provide adaptive, memory-free conditioning. Instead of retrieving references, PDM learns compact visual prototypes from clean features via contrastive learning, then aligns noisy representations with semantically relevant patterns during denoising. Experiments demonstrate that PDM sustains high generation quality while lowering computational and storage costs, offering a scalable alternative to retrieval-based conditioning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。