用结构化提示增强推荐排序，缓解大模型对位置偏见的敏感性
Prompt-Based LLMs for Position Bias-Aware Reranking in Personalized Recommendations
- 混合传统模型与大模型，用结构化提示重排前k个候选项
- 随机化用户历史可提升排序质量，但大模型重排未超越基础模型
- 明确指令无法有效减轻位置偏差，暴露大模型建模能力局限
推荐系统通过建模用户偏好与行为，在数字平台中实现个性化内容推送。近期，大语言模型（LLMs）因无需任务特定训练即可生成个性化输出，被用于基于提示的推荐。然而，现有方法存在上下文窗口有限、点对点与成对提示效率低、列表级排序受令牌约束等问题。此外，大模型对位置偏差敏感，可能过度关注提示中靠前的项目，忽视其真实相关性。为此，我们提出一种混合框架，结合传统推荐模型与大模型，利用结构化提示对前k个候选项进行重排。在MovieLens-100K数据集上评估用户历史重排与指令提示对缓解位置偏差的效果。结果表明，随机化用户历史可提升排名质量，但基于大模型的重排未能优于基线模型；明确指示减少位置偏差也未见效。实验揭示了大模型在建模排序上下文与缓解偏差方面的局限性。代码已公开于https://github.com/aminul7506/LLMForReRanking。
原文摘要 · Abstract (English)
Recommender systems are essential for delivering personalized content across digital platforms by modeling user preferences and behaviors. Recently, large language models (LLMs) have been adopted for prompt-based recommendation due to their ability to generate personalized outputs without task-specific training. However, LLM-based methods face limitations such as limited context window size, inefficient pointwise and pairwise prompting, and difficulty handling listwise ranking due to token constraints. LLMs can also be sensitive to position bias, as they may overemphasize earlier items in the prompt regardless of their true relevance. To address and investigate these issues, we propose a hybrid framework that combines a traditional recommendation model with an LLM for reranking top-k items using structured prompts. We evaluate the effects of user history reordering and instructional prompts for mitigating position bias. Experiments on MovieLens-100K show that randomizing user history improves ranking quality, but LLM-based reranking does not outperform the base model. Explicit instructions to reduce position bias are also ineffective. Our evaluations reveal limitations in LLMs' ability to model ranking context and mitigate bias. Our code is publicly available at https://github.com/aminul7506/LLMForReRanking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。