通过分组渐进式学习，让大模型更精准地适应用户偏好。
PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptation
- 按用户偏好分组，分阶段逐步个性化大模型。
- 在多个任务上优于当前最优模型，提升显著。
- 适合需要高效个性化服务的场景，如智能助手。
个性化大语言模型旨在根据用户偏好调整输出。近期基于参数高效微调（PEFT）的方法通过用户历史数据微调特定参数，实现从通用模型到个性化模型的适配。然而，用户数据通常稀疏，难以捕捉个体模式。为此，我们提出PROgressive PERsonalization（PROPER），一种受社会学中观层次理论启发的渐进式学习框架。PROPER通过用户偏好分组，连接群体级与用户级模型，并分阶段进行适配。该框架结合混合专家（MoE）结构与低秩适配（LoRA），采用用户感知路由自动分配用户至合适组别；同时提出一种LoRA感知路由，促进个体用户LoRA与组级LoRA的融合。实验结果表明，PROPER在多个任务上显著优于当前最优模型，验证了方法的有效性。
原文摘要 · Abstract (English)
Personalized large language models (LLMs) aim to tailor their outputs to user preferences. Recent advances in parameter-efficient fine-tuning (PEFT) methods have highlighted the effectiveness of adapting population-level LLMs to personalized LLMs by fine-tuning user-specific parameters with user history. However, user data is typically sparse, making it challenging to adapt LLMs to specific user patterns. To address this challenge, we propose PROgressive PERsonalization (PROPER), a novel progressive learning framework inspired by meso-level theory in social science. PROPER bridges population-level and user-level models by grouping users based on preferences and adapting LLMs in stages. It combines a Mixture-of-Experts (MoE) structure with Low Ranked Adaptation (LoRA), using a user-aware router to assign users to appropriate groups automatically. Additionally, a LoRA-aware router is proposed to facilitate the integration of individual user LoRAs with group-level LoRAs. Experimental results show that PROPER significantly outperforms SOTA models across multiple tasks, demonstrating the effectiveness of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。