arXiv:2606.04547cs.IRcs.CL2026-06被引 1

用轻量级嵌入表示用户偏好,实现高效可扩展的LLM个性化。

Beyond Retrieval: Learning Compact User Representations for Scalable LLM Personalization

论文配图:Beyond Retrieval: Learning Compact User Representations for Scalable LLM Personalization
图 1 · 摘自论文原文
  • 将用户偏好建模为可学习的前缀嵌入,替代传统提示词构造
  • 在6个任务上优于基线方法,1000用户时参数量仅为基准的1/130
  • 适合需要大规模用户个性化的场景,如智能助手、推荐系统

个性化大语言模型需在保持鲁棒性和部署效率的前提下适配个体用户行为。现有方法或在输入层通过检索用户历史构建提示词,或在参数层维护用户专属的高效模块,前者依赖检索质量与提示设计,后者随用户数增长带来存储与维护成本。为此,本文提出TAP-PER(Temporal Attentive Prefix for PERsonalization),一种基于前缀的框架,将用户偏好编码为可学习的表示,无需显式提示构造,并以轻量级用户状态前缀嵌入替代重型用户适配器。受个性化推荐系统启发,TAP-PER将用户建模分解为用户状态与查询条件两部分,并引入时间信号捕捉兴趣演变。在六个LaMP任务上的实验表明,TAP-PER在分类、评分和生成任务中均持续优于提示法与模型法基线。此外,在1000用户规模下,其每用户参数量仅为OPPU的1/130,总参数量约为PER-PCS的一半,证明无需显式提示或重型适配器即可实现可扩展的LLM个性化。

原文摘要 · Abstract (English)

Personalizing large language models requires adapting model behavior to individual users while preserving robustness and deployment-scale efficiency. Existing approaches typically personalize LLMs either at the input level, by retrieving user histories or constructing profile prompts, or at the parameter level, by maintaining user-specific parameter-efficient modules. The former makes personalization sensitive to retrieval quality and prompt design, whereas the latter incurs storage and maintenance costs that grow with the user population. To address these limitations, we propose TAP-PER (Temporal Attentive Prefix for PERsonalization), a prefix-based framework that encodes user preferences as learnable representations, eliminating explicit prompt construction and replacing heavy per-user adapters with lightweight user-state prefix embeddings. Inspired by personalized recommendation systems, TAP-PER decomposes user modeling into user-state and query-conditioned components, and incorporates temporal signals to capture the evolving nature of user interests. Experiments on six LaMP tasks show that TAP-PER consistently outperforms prompt-based and model-based baselines across classification, rating, and generation settings. Moreover, TAP-PER uses 130x fewer per-user parameters than OPPU and roughly half the total parameter footprint of PER-PCS at the 1,000-user scale, demonstrating that scalable LLM personalization can be achieved without explicit prompt construction or heavy per-user adapters.

大模型个性化前缀学习高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。