用动态贝叶斯方法让大模型推荐更准更公平,突破传统静态优化瓶颈。
Beyond Static Best-of-N: Bayesian List-wise Alignment for LLM-based Recommendation

- 基于贝叶斯框架动态更新推荐目标,自适应模型进化能力
- 在三个真实数据集上超越现有方法,提升召回率、NDCG和公平性
- 适合追求高阶推荐质量的工业场景,尤其关注多样性与公平性的应用
大语言模型通过生成能力革新了推荐系统(LLM4Rec),但现有方法多依赖词元级目标,难以优化列表级及不可微指标(如NDCG、公平性)——这些正是推荐质量的核心。虽然推理阶段的Best-of-N(BoN)可直接优化这些指标,但其计算成本过高,难部署。为解决此问题,BoN对齐旨在将搜索能力蒸馏进模型,但现有方法存在两大缺陷:(1)无差别监督——静态参考无法区分超出其经验范围的候选项质量,导致排序引导失效;(2)梯度衰减——随着策略演进,有效监督信号迅速衰减,优化效率低下。为此,我们提出BLADE(Bayesian List-wise Alignment via Dynamic Estimation)。不同于静态方法,BLADE引入贝叶斯框架,融合历史先验与模型当前回滚结果的动态证据,持续更新目标分布。该机制构建出随模型能力自演化的目标,确保训练信号全程有效。在三个真实数据集上的大量实验表明,BLADE显著优于现有最优基线,突破静态性能上限,在排名准确率(召回率、NDCG)和复杂列表级指标(公平性、多样性)上均实现持续提升。代码已开源:https://github.com/RegionCh/BLADE。
原文摘要 · Abstract (English)
Large Language Models have revolutionized recommender systems (LLM4Rec) by leveraging their generative capabilities to model complex user preferences. However, existing LLM4Rec methods primarily rely on token-level objectives, making it difficult to optimize list-level and non-differentiable metrics (e.g., NDCG, fairness) that define actual recommendation quality. While Best-of-N (BoN) directly optimizes these metrics during inference, its high computational cost hinders real-world deployment. To address this, BoN Alignment aims to distill the search capability into the model itself, yet current approaches suffer from two critical limitations: (1) Indiscriminate Supervision, where the static reference fails to distinguish the relative quality of candidates exceeding its empirical range, leading to a loss of ranking guidance; and (2) Gradient Decay, where the effective supervision signal rapidly diminishes as the evolving policy improves, resulting in inefficient optimization. To overcome these challenges, we propose BLADE (Bayesian List-wise Alignment via Dynamic Estimation). Unlike static approaches, BLADE introduces a Bayesian framework that continuously updates the target distribution by fusing historical priors with dynamic evidence from the model's current rollouts. This mechanism constructs a self-evolving target that adapts to the model's growing capabilities, ensuring the training signal remains informative throughout the learning process. Extensive experiments on three real-world datasets demonstrate that BLADE significantly outperforms state-of-the-art baselines. Crucially, it breaks the static performance upper bound, achieving sustained gains in both ranking accuracy (Recall, NDCG) and complex list-wise metrics (Fairness, Diversity). The code is available via https://github.com/RegionCh/BLADE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。