arXiv:2503.22352cs.CV2025-03被引 1

用元学习让LoRA更懂特定身份,少样本也能精准复现人物形象。

Meta-LoRA: Meta-Learning LoRA Components for Domain-Aware ID Personalization

  • 分三层的LoRA结构,先学通用特征再专精个体
  • 仅需少量参考图即可快速适配,生成图像身份一致率更高
  • 适合需要快速个性化生成的场景,如数字人、虚拟偶像

文本到图像生成模型,尤其是潜在扩散模型(LDMs),在根据文本提示生成高质量图像方面取得了显著进展。然而,实现身份个性化——即在仅有少量参考图像的情况下持续生成特定主体的输出——仍是核心挑战。为此,我们提出元低秩适应(Meta-LoRA),一种利用元学习将领域特定先验编码进基于LoRA的身份个性化框架。该方法引入一种结构化的三层LoRA架构,将无关身份的知识与特定身份的适配分离。第一阶段,通过多个主体的元训练,使LoRA Meta-Down层学习共享流形,捕捉通用身份相关特征;第二阶段,仅优化LoRA-Mid和LoRA-Up层以针对特定主体进行专业化,大幅降低适配时间并提升身份保真度。为评估该方法,我们构建了新的基准数据集Meta-PHD,并与现有先进方法对比。结果表明,Meta-LoRA在身份保留、计算效率和跨身份条件下的适应性方面均表现优异。代码、模型权重和数据集已公开于barisbatuhan.github.io/Meta-LoRA。

原文摘要 · Abstract (English)

Recent advancements in text-to-image generative models, particularly latent diffusion models (LDMs), have demonstrated remarkable capabilities in synthesizing high-quality images from textual prompts. However, achieving identity personalization-ensuring that a model consistently generates subject-specific outputs from limited reference images-remains a fundamental challenge. To address this, we introduce Meta-Low-Rank Adaptation (Meta-LoRA), a novel framework that leverages meta-learning to encode domain-specific priors into LoRA-based identity personalization. Our method introduces a structured three-layer LoRA architecture that separates identity-agnostic knowledge from identity-specific adaptation. In the first stage, the LoRA Meta-Down layers are meta-trained across multiple subjects, learning a shared manifold that captures general identity-related features. In the second stage, only the LoRA-Mid and LoRA-Up layers are optimized to specialize on a given subject, significantly reducing adaptation time while improving identity fidelity. To evaluate our approach, we introduce Meta-PHD, a new benchmark dataset for identity personalization, and compare Meta-LoRA against state-of-the-art methods. Our results demonstrate that Meta-LoRA achieves superior identity retention, computational efficiency, and adaptability across diverse identity conditions. Our code, model weights, and dataset are released on barisbatuhan.github.io/Meta-LoRA.

身份个性化LoRA元学习图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。