arXiv:2607.11354cs.IR2026-07

用大模型补全用户偏好,让离线推荐评估更准

User Preference Induction with LLMs for Offline Top-N Recommendation Evaluation

  • 用大模型生成用户兴趣文本,补全未标注的偏好
  • 在多推荐器候选集上扩展评判,提升评估完整性
  • 适合关注评估公平性与冷启动推荐的研究者

离线评估是推荐系统比较的标准方法,但依赖不完整的相关性信息。多数基准数据集中仅观察到少量用户-物品偏好,未评分项常被默认为非相关,这一缺失即负假设会引入偏差,惩罚无反馈的合理推荐,并偏爱热门或高曝光项目。本文提出基于大语言模型的框架,用于扩展离线推荐评估中的相关性判断。该方法分两步:首先将用户历史交互转化为文本兴趣档案;其次,基于该档案,由大模型对原始测试数据中无标签的候选物品进行相关性判断。为保证可操作性和评估聚焦性,评判扩展应用于由多个推荐器输出的顶级候选池。由此生成的丰富判断提供了先前未观测的用户-物品对的相关证据,使排序指标可在更完整的基础上计算。实验表明,该方法能有效提升离线Top-N评估的鲁棒性,缓解稀疏反馈带来的流行度敏感偏差。

原文摘要 · Abstract (English)

Offline evaluation is the standard methodology for comparing top-N recommender systems, yet it relies on incomplete relevance information. In most benchmark datasets, only a small subset of user--item preferences is observed, and unjudged items are commonly treated as non-relevant. This missing-as-negative assumption can bias evaluation, penalize plausible recommendations with no recorded feedback, and favour algorithms that concentrate on popular or highly exposed items. We propose an LLM-based framework to expand relevance judgements for offline recommender evaluation. Our approach uses large language models in two complementary roles. First, a preference induction stage summarizes each user's historical interactions into a textual profile that captures their tastes and interests. Second, conditioned on this profile, an LLM acts as a relevance judge for candidate recommended items that lack observed labels in the original test data. To make this process tractable and evaluation-focused, we apply judgement expansion to a pooled candidate set built from the top-ranked outputs of multiple recommenders. The resulting enriched judgements provide additional relevance evidence for previously unobserved user--item pairs, enabling ranking metrics to be computed on a more complete basis. Experimental results show that this approach is a promising strategy for improving the robustness of offline top-N evaluation and mitigating the popularity-sensitive distortions caused by sparse feedback.

推荐系统大模型应用离线评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。