无需微调，用缓存特征实现高效个性化图像生成
DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching
- 缓存参考图像的少量特征，通过轻量适配器动态调节生成
- 参数量少一个数量级，图像与文本对齐效果达当前最佳
- 适合快速部署、资源受限场景的个性化图像生成任务
个性化图像生成需要文本到图像模型捕捉参考主体的核心特征，以在不同上下文中进行可控生成。现有方法因训练复杂、推理成本高、灵活性差或其组合而面临挑战。本文提出DreamCache，一种可扩展的高效个性化图像生成方法。通过缓存预训练扩散去噪器中部分层和单个时间步的少量参考图像特征，DreamCache利用轻量级训练好的条件适配器动态调制生成图像特征。该方法在仅使用一个数量级更少的额外参数情况下，实现了最先进的图像与文本对齐效果，且在计算效率和通用性上优于现有模型。
原文摘要 · Abstract (English)
Personalized image generation requires text-to-image generative models that capture the core features of a reference subject to allow for controlled generation across different contexts. Existing methods face challenges due to complex training requirements, high inference costs, limited flexibility, or a combination of these issues. In this paper, we introduce DreamCache, a scalable approach for efficient and high-quality personalized image generation. By caching a small number of reference image features from a subset of layers and a single timestep of the pretrained diffusion denoiser, DreamCache enables dynamic modulation of the generated image features through lightweight, trained conditioning adapters. DreamCache achieves state-of-the-art image and text alignment, utilizing an order of magnitude fewer extra parameters, and is both more computationally effective and versatile than existing models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。