用低秩分解压缩个性化图像生成模型,省空间又不丢质量。
Delta-SVD: Efficient Compression for Personalized Text-to-Image Models
- 对微调产生的参数增量做奇异值分解,提取低秩结构。
- 压缩率高,生成质量损失极小(CLIP、SSIM、FID接近原模型)。
- 无需重训练,可直接部署,适合大规模个性化应用。
个性化文生图模型如DreamBooth需微调大规模扩散模型,导致存储开销大。本文提出Delta-SVD,一种无需训练的后处理压缩方法,针对DreamBooth微调引发的参数增量。关键观察是:这些增量因个性化稀疏局部特性,具有强低秩结构。该方法先对权重增量进行奇异值分解(SVD),再通过基于能量的秩截断策略,在压缩效率与重建保真度间取得平衡。压缩后的模型可即插即用,推理时实时重构。方法简单高效,不改变原架构。在多主体数据集上的实验表明,Delta-SVD实现显著压缩,生成质量损失可忽略(以CLIP分数、SSIM和FID衡量)。该方法使个性化扩散模型可扩展、高效部署,适用于需存储和运行大量定制化模型的实际场景。
原文摘要 · Abstract (English)
Personalized text-to-image models such as DreamBooth require fine-tuning large-scale diffusion backbones, resulting in significant storage overhead when maintaining many subject-specific models. We present Delta-SVD, a post-hoc, training-free compression method that targets the parameter weights update induced by DreamBooth fine-tuning. Our key observation is that these delta weights exhibit strong low-rank structure due to the sparse and localized nature of personalization. Delta-SVD first applies Singular Value Decomposition (SVD) to factorize the weight deltas, followed by an energy-based rank truncation strategy to balance compression efficiency and reconstruction fidelity. The resulting compressed models are fully plug-and-play and can be re-constructed on-the-fly during inference. Notably, the proposed approach is simple, efficient, and preserves the original model architecture. Experiments on a multiple subject dataset demonstrate that Delta-SVD achieves substantial compression with negligible loss in generation quality measured by CLIP score, SSIM and FID. Our method enables scalable and efficient deployment of personalized diffusion models, making it a practical solution for real-world applications that require storing and deploying large-scale subject customizations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。