自动分离风格与噪声，让个性化图像生成更高效。
AC-LoRA: Auto Component LoRA for Personalized Artistic Style Image Generation
- 基于SVD和动态启发式方法自动分解LoRA矩阵成分
- 在多项指标上平均提升9%性能，缓解过拟合与欠拟合
- 适合需要快速定制艺术风格的创作者或应用开发者
个性化图像生成允许用户保留少量参考图像中的风格或主体特征以进行后续生成。随着大规模文本到图像模型的发展，许多技术被提出以高效微调这些模型实现个性化，例如低秩适配（LoRA）。然而，基于LoRA的方法常面临调整秩参数以获得理想效果的挑战。为此，本文提出自动组件LoRA（AC-LoRA），能够通过奇异值分解（SVD）与动态启发式策略，在训练过程中自动分离LoRA矩阵中的信号成分与噪声成分，从而实现快速高效的个性化艺术风格图像生成。该方法在克服模型欠拟合或过拟合问题方面表现优异。在FID、CLIP、DINO和ImageReward等指标上验证，平均性能提升达9%。
原文摘要 · Abstract (English)
Personalized image generation allows users to preserve styles or subjects of a provided small set of images for further image generation. With the advancement in large text-to-image models, many techniques have been developed to efficiently fine-tune those models for personalization, such as Low Rank Adaptation (LoRA). However, LoRA-based methods often face the challenge of adjusting the rank parameter to achieve satisfactory results. To address this challenge, AutoComponent-LoRA (AC-LoRA) is proposed, which is able to automatically separate the signal component and noise component of the LoRA matrices for fast and efficient personalized artistic style image generation. This method is based on Singular Value Decomposition (SVD) and dynamic heuristics to update the hyperparameters during training. Superior performance over existing methods in overcoming model underfitting or overfitting problems is demonstrated. The results were validated using FID, CLIP, DINO, and ImageReward, achieving an average of 9% improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。