用自适应网络生成LoRA权重，零样本实现高保真人像合成
HyperLoRA: Parameter-Efficient Adaptive Generation for Portrait Synthesis
- 设计自适应插件网络动态生成LoRA参数，无需微调
- 支持单图或多图输入，零样本生成高保真人像
- 兼顾真实感与可编辑性，适合个性化内容创作
个性化人像合成在社交娱乐等领域具有重要意义。基于逐人微调的方法（如LoRA、DreamBooth）虽能生成逼真人像，但需对每个个体进行训练，耗时耗资源且存在不稳定风险。基于适配器的技术（如IP-Adapter）冻结主模型参数，采用插件结构实现零样本推理，但往往缺乏自然感与真实性，这在人像合成任务中尤为关键。本文提出一种参数高效自适应生成方法HyperLoRA，通过自适应插件网络生成LoRA权重，融合了LoRA的高性能与适配器的零样本能力。通过精心设计的网络结构与训练策略，实现了零样本个性化人像生成（支持单图及多图输入），具备高保真度、真实感与可编辑性。
原文摘要 · Abstract (English)
Personalized portrait synthesis, essential in domains like social entertainment, has recently made significant progress. Person-wise fine-tuning based methods, such as LoRA and DreamBooth, can produce photorealistic outputs but need training on individual samples, consuming time and resources and posing an unstable risk. Adapter based techniques such as IP-Adapter freeze the foundational model parameters and employ a plug-in architecture to enable zero-shot inference, but they often exhibit a lack of naturalness and authenticity, which are not to be overlooked in portrait synthesis tasks. In this paper, we introduce a parameter-efficient adaptive generation method, namely HyperLoRA, that uses an adaptive plug-in network to generate LoRA weights, merging the superior performance of LoRA with the zero-shot capability of adapter scheme. Through our carefully designed network structure and training strategy, we achieve zero-shot personalized portrait generation (supporting both single and multiple image inputs) with high photorealism, fidelity, and editability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。