用自然语言做个性化接口，让模型偏好可读可迁移。
Text as a Universal Interface for Transferable Personalization
- 用文本描述用户偏好，替代黑箱参数表示。
- 8B模型在9个基准上超越更大开源模型，跨任务迁移强。
- 适合需要可解释个性化的应用，如对话系统、内容生成。
我们研究大语言模型中的个性化问题。以往方法将用户偏好表示为隐式的、模型专属的向量或参数，形成难以理解且无法跨模型、跨任务迁移的黑箱配置。本文主张以自然语言作为通用、与模型和任务无关的偏好表达接口。该方法实现可解释、可复用的偏好描述，并能随新交互持续演化。为此，我们提出两阶段训练框架：先在高质量合成数据上进行监督微调，再通过强化学习优化长期效用与跨任务可迁移性。基于此框架，我们开发了AlignXplore+——一个生成文本化偏好摘要的通用偏好推理模型。在九个基准上的实验表明，我们的8B模型性能达到顶尖水平，显著优于更大规模的开源模型，且在不同任务、模型族和交互格式间展现出强大迁移能力。
原文摘要 · Abstract (English)
We study the problem of personalization in large language models (LLMs). Prior work predominantly represents user preferences as implicit, model-specific vectors or parameters, yielding opaque ``black-box'' profiles that are difficult to interpret and transfer across models and tasks. In contrast, we advocate natural language as a universal, model- and task-agnostic interface for preference representation. The formulation leads to interpretable and reusable preference descriptions, while naturally supporting continual evolution as new interactions are observed. To learn such representations, we introduce a two-stage training framework that combines supervised fine-tuning on high-quality synthesized data with reinforcement learning to optimize long-term utility and cross-task transferability. Based on this framework, we develop AlignXplore+, a universal preference reasoning model that generates textual preference summaries. Experiments on nine benchmarks show that our 8B model achieves state-of-the-art performanc -- outperforming substantially larger open-source models -- while exhibiting strong transferability across tasks, model families, and interaction formats.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。