用少量模型组合实现个性化，让每个用户都能找到匹配的AI助手。
Many Preferences, Few Policies: Towards Scalable Language Model Personalization
- 通过多维偏好向量建模用户需求，生成少量高代表性的模型组合。
- 理论证明组合规模与个性化效果的平衡关系，支持任意用户偏好调整。
- 适合追求高效个性化服务的系统设计者，尤其关注模型部署成本。
大语言模型个性化的目标是为每位用户提供一个完全契合其偏好的专属模型。然而，为每个用户维护独立模型在计算、内存和系统复杂度上不可行。本文提出一种系统性方法，选择少量代表性模型构成模型组合，以覆盖异构用户的多样化偏好。通过多维度权重向量建模用户在安全、幽默、简洁等特质上的偏好，并基于各维度奖励函数，算法PALM（对齐模型组合）生成小规模模型集合，使得对于任意权重向量，该组合中均包含对应标量化目标下的近优模型。据我们所知，这是首个在组合大小与近似质量上提供理论保证的LLM个性化方法，揭示了系统成本与个性化程度之间的权衡，以及覆盖用户偏好分布所需的模型多样性。实验验证了这些理论保证，并展示了比主流基线更强的输出多样性。
原文摘要 · Abstract (English)
The holy grail of LLM personalization is a single LLM for each user, perfectly aligned with that user's preferences. However, maintaining a separate LLM per user is impractical due to constraints on compute, memory, and system complexity. We address this challenge by developing a principled method for selecting a small portfolio of LLMs that captures representative behaviors across heterogeneous users. We model user preferences across multiple traits (e.g., safety, humor, brevity) through a multi-dimensional weight vector. Given reward functions across these dimensions, our algorithm PALM (Portfolio of Aligned LLMs) generates a small portfolio of LLMs such that, for any weight vector, the portfolio contains a near-optimal LLM for the corresponding scalarized objective. To the best of our knowledge, this is the first result that provides theoretical guarantees on both the size and approximation quality of LLM portfolios for personalization. It characterizes the trade-off between system cost and personalization, as well as the diversity of LLMs required to cover the landscape of user preferences. We provide empirical results that validate these guarantees and demonstrate greater output diversity over common baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。