不用训练,靠结构先验实现高效冷启动个性化推荐
Cold-Start Personalization via Training-Free Priors from Structured World Models
- 离线构建偏好关联的世界模型,线上用贝叶斯推理选问题
- 仅需3-5次交互即达80.8%偏好对齐,比强化学习高12.3个百分点
- 适合低资源场景,参数量仅1万级,适配各类下游任务
冷启动个性化需在无用户历史数据时通过交互推断偏好。核心挑战是路由问题:每个任务有数十个偏好维度,但每位用户只关心少数几个,且关注点因人而异。有限提问预算下无结构提问易遗漏关键维度。强化学习虽为自然框架,但在多轮设置中终端奖励无法利用偏好数据的因子化结构,导致策略退化为固定提问序列,忽略用户反馈。我们提出将冷启动信息获取分解为离线结构学习与在线贝叶斯推断。Pep(Preference Elicitation with Priors)离线从完整偏好档案中学习偏好相关性的结构化世界模型,线上进行无需训练的贝叶斯推断,选择信息量大的问题并预测完整偏好,包括从未询问的维度。该框架在下游求解器间模块化,仅需简单信念模型。在医疗、数学、社会及常识推理任务中,Pep生成响应与用户明示偏好的对齐率达80.8%,高于强化学习的68.5%;交互次数减少3-5倍。当两名用户对同一问题回答不同时,Pep有39-62%概率调整后续提问,而强化学习仅为0-28%。其参数量约1万,远低于强化学习的80亿,表明冷启动信息获取的瓶颈在于能否利用偏好数据的因子化结构。
原文摘要 · Abstract (English)
Cold-start personalization requires inferring user preferences through interaction when no user-specific historical data is available. The core challenge is a routing problem: each task admits dozens of preference dimensions, yet individual users care about only a few, and which ones matter depends on who is asking. With a limited question budget, asking without structure will miss the dimensions that matter. Reinforcement learning is the natural formulation, but in multi-turn settings its terminal reward fails to exploit the factored, per-criterion structure of preference data, and in practice learned policies collapse to static question sequences that ignore user responses. We propose decomposing cold-start elicitation into offline structure learning and online Bayesian inference. Pep (Preference Elicitation with Priors) learns a structured world model of preference correlations offline from complete profiles, then performs training-free Bayesian inference online to select informative questions and predict complete preference profiles, including dimensions never asked about. The framework is modular across downstream solvers and requires only simple belief models. Across medical, mathematical, social, and commonsense reasoning, Pep achieves 80.8% alignment between generated responses and users' stated preferences versus 68.5% for RL, with 3-5x fewer interactions. When two users give different answers to the same question, Pep changes its follow-up 39-62% of the time versus 0-28% for RL. It does so with ~10K parameters versus 8B for RL, showing that the bottleneck in cold-start elicitation is the capability to exploit the factored structure of preference data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。