用用户嵌入生成软前缀，零成本实现大模型个性化
Embedding-to-Prefix: Parameter-Efficient Personalization for Pre-Trained Large Language Models
- 将用户嵌入通过投影转为软前缀注入模型
- 在多个数据集上达到接近微调的个性化效果
- 适合需要快速部署个性化的实际场景
大语言模型在生成上下文相关内容方面表现优异，但将其输出适配到具体用户以实现有效个性化仍具挑战。尽管用户偏好或行为通常已有预训练的用户嵌入表示，现有方法往往依赖昂贵的微调或冗长的提示。我们提出Embedding-to-Prefix（E2P），一种参数高效的个性化方法：通过学习一个投影，将预计算的用户嵌入注入到LLM的隐藏表示空间，转化为单一软前缀。该方法保持主干模型冻结，避免了高成本的适应操作。我们在两个公开数据集和一个生产环境进行了评估：在Persona-Chat上进行对话个性化，在PENS上生成上下文标题，并在音乐与播客消费场景中实现大规模个性化。结果表明，E2P能有效保留上下文信号，以极低计算开销实现优异性能，为生成式AI系统提供了一种可扩展、高效的上下文定制方案。
原文摘要 · Abstract (English)
Large language models (LLMs) excel at generating contextually relevant content. However, tailoring these outputs to individual users for effective personalization is a significant challenge. While rich user-specific information often exists as pre-existing user representations, such as embeddings learned from preferences or behaviors, current methods to leverage these for LLM personalization typically require costly fine-tuning or token-heavy prompting. We propose Embedding-to-Prefix (E2P), a parameter-efficient method that injects pre-computed context embeddings into an LLM's hidden representation space through a learned projection to a single soft token prefix. This enables effective personalization while keeping the backbone model frozen and avoiding expensive adaptation techniques. We evaluate E2P across two public datasets and in a production setting: dialogue personalization on Persona-Chat, contextual headline generation on PENS, and large-scale personalization for music and podcast consumption. Results show that E2P preserves contextual signals and achieves strong performance with minimal computational overhead, offering a scalable, efficient solution for contextualizing generative AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。