arXiv:2604.16318cs.IRcs.CL2026-04被引 1

LLM重排器在冷启动推荐中表现差,因候选覆盖少、曝光集中且区分度低。

Diagnosing LLM-based Rerankers in Cold-Start Recommender Systems: Coverage, Exposure and Practical Mitigations

论文配图:Diagnosing LLM-based Rerankers in Cold-Start Recommender Systems: Coverage, Exposure and Practical Mitigations
图 1 · 摘自论文原文
  • 通过控制实验诊断出三类失效模式:候选覆盖率低、曝光严重不均、得分区分弱。
  • 主流基线在命中率上远超LLM重排器(HR@10: 0.268 vs. 0.008),差距显著。
  • 适合关注冷启动推荐系统落地问题的研究者与工程师参考。

大型语言模型(LLMs)和交叉编码器重排器被用于提升推荐系统,尤其在用户交互历史有限的冷启动场景。然而实际部署中,基于LLM的方法与简单基线间存在显著性能差距。本文针对冷启动电影推荐,在Serendipity-2018数据集上对交叉编码器重排器进行系统性诊断。通过对500名用户、多随机种子的受控实验,发现三大关键失效模式:(1)候选生成阶段检索覆盖率极低(recall@200 = 0.109,基线为0.609);(2)重排器严重偏向少数物品,仅推荐3个唯一物品,而随机基线可覆盖497个;(3)相关与无关项得分差异微弱(均值差=0.098,Cohen's d=0.13)。结果显示,基于流行度的排序显著优于LLM重排(HR@10: 0.268 vs. 0.008,p < 0.001),性能差距主要源于检索阶段限制,而非重排器能力。据此提出混合检索策略、候选池优化与分数校准等实用建议。所有代码、配置与结果均公开以保证可复现性。

原文摘要 · Abstract (English)

Large language models (LLMs) and cross-encoder rerankers have gained attention for improving recommender systems, particularly in cold-start scenarios where user interaction history is limited. However, practical deployment reveals significant performance gaps between LLM-based approaches and simple baselines. This paper presents a systematic diagnostic study of cross-encoder rerankers in cold-start movie recommendation using the Serendipity-2018 dataset. Through controlled experiments with 500 users across multiple random seeds, we identify three critical failure modes: (1) low retrieval coverage in candidate generation (recall@200 = 0.109 vs. 0.609 for baselines), (2) severe exposure bias with rerankers concentrating recommendations on 3 unique items versus 497 for random baseline, and (3) minimal score discrimination between relevant and irrelevant items (mean difference = 0.098, Cohen's d = 0.13). We demonstrate that popularity-based ranking substantially outperforms LLM reranking (HR@10: 0.268 vs. 0.008, p < 0.001), with the performance gap primarily attributable to retrieval stage limitations rather than reranker capacity. Based on these findings, we provide actionable recommendations including hybrid retrieval strategies, candidate pool size optimization, and score calibration techniques. All code, configurations, and experimental results are made available for reproducibility.

冷启动重排器推荐系统LLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。