用共享空间+低秩调制，实现超省参数的个性化大模型
PLUME: Parameter-Efficient Personalization of Large Language Models via Low-Rank User Modulation in Shared Subspaces
- 在共享任务子空间中训练小矩阵，实现用户专属模型
- 相比基线减少95%以上用户参数，性能相当或更优
- 适合大规模个性化场景，尤其资源受限时
个性化大型语言模型对匹配用户风格、意图和偏好至关重要。尽管针对每个用户的微调能显著提升个性化质量，但会带来巨大参数与存储开销，限制其在大规模用户群体中的应用。我们提出PLUME（通过用户调制与共享子空间实现的个性化低秩适配），一种轻量级框架，通过利用共享的任务特定子空间实现高效且富有表现力的用户定制。具体而言，PLUME首先从聚合用户数据中学习全局任务子空间。个性化通过在此子空间内仅训练一个轻量级的小方阵实现，使每位用户获得定制化模型的同时保持共享组件固定。进一步引入跨层共享参数与秩1残差项，大幅降低冗余并维持表达能力。在多个个性化文本生成基准上实验表明，PLUME性能媲美甚至优于强基线，同时将每用户参数减少超过95%。这些结果确立了以最小残差进行共享子空间调制是一种可扩展且语义合理的大型语言模型个性化方法。
原文摘要 · Abstract (English)
Personalizing large language models (LLMs) is essential for delivering AI assistance that aligns with individual users' styles, intents, and preferences. While per-user fine-tuning can substantially enhance personalization quality, it introduces significant parameter and storage overhead, limiting scalability to large user populations. We propose PLUME (Personalized Low-Rank Adaptation through User Modulation and Shared Subspace), a lightweight framework that achieves efficient and expressive per-user adaptation by leveraging a shared task-specific subspace. Specifically, PLUME first learns a global task subspace from aggregated user data. Personalization is then achieved by training only a lightweight small square matrix within this subspace, enabling each user to obtain a tailored model while keeping shared components fixed. Cross-layer shared parameters and rank-1 residual terms are further introduced to significantly reduce redundancy while maintaining expressiveness. Experiments on multiple personalized text generation benchmarks demonstrate that PLUME achieves comparable or superior performance to strong baselines, while reducing per-user parameters by over 95%. These results establish shared-subspace modulation with minimal residuals as a scalable and semantically grounded approach to LLM personalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。