LLM推荐存在位置偏差，新方法RISE可不调参提升推荐稳定性
Evaluating Position Bias in Large Language Model Recommendations
- 提出迭代选择提示策略RISE，缓解候选列表顺序对推荐结果的影响
- 实验显示该方法显著降低输入顺序敏感性，提升推荐一致性
- 无需微调或后处理，适合快速部署的零样本推荐场景
大型语言模型（LLMs）正被广泛探索用于推荐任务，具备零样本和指令遵循能力，无需特定任务训练。然而，直接应用于推荐时存在重要隐患。本文揭示了基于LLM的推荐模型存在位置偏差，即提示中候选项的排列顺序会显著影响推荐结果。我们在真实数据集上分析了这一偏差，发现LLM对输入顺序高度敏感。为此，我们提出一种新的提示策略——基于迭代选择的排序（Ranking via Iterative SElection, RISE），在多个基准数据集上与多种基线方法对比。实验表明，该方法有效降低了对输入顺序的敏感性，提升了推荐稳定性，且无需模型微调或后处理。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are being increasingly explored as general-purpose tools for recommendation tasks, enabling zero-shot and instruction-following capabilities without the need for task-specific training. While the research community is enthusiastically embracing LLMs, there are important caveats to directly adapting them for recommendation tasks. In this paper, we show that LLM-based recommendation models suffer from position bias, where the order of candidate items in a prompt can disproportionately influence the recommendations produced by LLMs. First, we analyse the position bias of LLM-based recommendations on real-world datasets, where results uncover systemic biases of LLMs with high sensitivity to input orders. Furthermore, we introduce a new prompting strategy to mitigate the position bias of LLM recommendation models called Ranking via Iterative SElection (RISE). We compare our proposed method against various baselines on key benchmark datasets. Experiment results show that our method reduces sensitivity to input ordering and improves stability without requiring model fine-tuning or post-processing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。