用自然语言摘要个性化大模型，省上下文还更准。
POPI: Personalizing LLMs via Optimized Natural Language Preference Inference
- 用自然语言接口提取用户偏好摘要，统一优化生成与摘要
- 在四个评测中提升个性化效果，上下文开销减少近十倍
- 适配冻结模型和商业API，可复用摘要实现跨场景个性化
大型语言模型通常对群体偏好进行对齐，但用户间存在显著差异。本文提出POPI，一种基于自然语言接口的用户级个性化框架，将问题分解为两个组件：共享的推理模型将异构用户信号提炼为简洁偏好摘要，共享生成器则根据该摘要生成个性化回复。两者在统一的偏好优化目标下训练，强化学习处理不可导的推理步骤。该目标可分解为生成近似误差与摘要信息量，揭示单一损失如何同时驱动精准生成与有效摘要。由于接口为自然语言，学习到的摘要可一次性获取并复用于不同生成器——包括冻结模型和黑箱商业API。在四个个性化基准测试中，POPI普遍提升个性化质量，上下文开销最高降低一个数量级。
原文摘要 · Abstract (English)
Large language models (LLMs) are typically aligned with population-level preferences, despite substantial variation across individual users. We introduce POPI, a user-level personalization framework that separates the problem into two components connected by a natural-language interface: a shared inference model that distills heterogeneous user signals into a concise preference summary, and a shared generator that conditions on this summary to produce personalized responses. Both components are trained under a unified preference-optimization objective, with reinforcement learning handling the non-differentiable inference step. This objective decomposes into generator approximation error and summary informativeness, revealing how a single loss simultaneously drives accurate generation and informative summarization. Because the interface is natural language, learned summaries can be inferred once per user and reused across different generators -- including frozen, black-box commercial APIs. Across four personalization benchmarks, POPI generally improves personalization quality while reducing context overhead by up to an order of magnitude.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。