用融合+自适应方法实现低成本个性化大模型,支持少样本快速适配。
MTA: A Merge-then-Adapt Framework for Personalized Large Language Model
- 先构建共享的元LoRA库,再动态融合相关模块生成用户专属参数。
- 在少样本场景下,通过叠加轻量级LoRA实现高效个性化,性能优于现有方法。
- 适合需要大规模个性化且存储受限的应用场景,如智能客服、个人助手。
个性化大语言模型(PLLM)旨在使模型输出符合个体用户偏好,对以用户为中心的应用至关重要。然而,当前为每位用户单独微调一个模块的方法存在两大局限:(1)存储成本随用户数线性增长,难以扩展;(2)从头微调静态模型在用户数据稀疏时表现不佳。为此,我们提出MTA框架,包含三个关键阶段:首先,通过选择锚点用户并预训练元个性化特征,在元LoRA模块中构建共享的元LoRA库;其次,引入自适应LoRA融合阶段,动态检索并合并最相关的锚点元LoRA,合成用户专属的参数,避免用户级存储,支持灵活组合;第三,提出少样本个性化中的LoRA堆叠机制,在融合后的LoRA上添加超低秩、轻量级的LoRA模块进行微调,实现少样本下的有效个性化。在LaMP基准上的大量实验表明,该方法在多个任务上超越现有最先进方法。
原文摘要 · Abstract (English)
Personalized Large Language Models (PLLMs) aim to align model outputs with individual user preferences, a crucial capability for user-centric applications. However, the prevalent approach of fine-tuning a separate module for each user faces two major limitations: (1) storage costs scale linearly with the number of users, rendering the method unscalable; and (2) fine-tuning a static model from scratch often yields suboptimal performance for users with sparse data. To address these challenges, we propose MTA, a Merge-then-Adapt framework for PLLMs. MTA comprises three key stages. First, we construct a shared Meta-LoRA Bank by selecting anchor users and pre-training meta-personalization traits within meta-LoRA modules. Second, to ensure scalability and enable dynamic personalization combination beyond static models, we introduce an Adaptive LoRA Fusion stage. This stage retrieves and dynamically merges the most relevant anchor meta-LoRAs to synthesize a user-specific one, thereby eliminating the need for user-specific storage and supporting more flexible personalization. Third, we propose a LoRA Stacking for Few-Shot Personalization stage, which applies an additional ultra-low-rank, lightweight LoRA module on top of the merged LoRA. Fine-tuning this module enables effective personalization under few-shot settings. Extensive experiments on the LaMP benchmark demonstrate that our approach outperforms existing SOTA methods across multiple tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。