用自对弈机制让大模型推荐更公平，不需额外数据
SPRec: Self-Play to Debias LLM-based Recommendation
- 通过迭代SFT+DPO，用模型自身预测结果做负样本
- 在多个数据集上提升推荐准确率与公平性
- 适合关注推荐系统偏见问题的研究者
大语言模型在推荐系统中受到广泛关注。现有方法多采用监督微调（SFT）适配推荐任务，但仅基于正样本会限制模型对用户偏好的对齐。近期提出的直接偏好优化（DPO）虽能显式对齐用户偏好，但我们发现其会加剧对少数物品的过度推荐，恶化信息茧房问题。本文提出SPRec，一种新颖的自对弈框架，无需额外数据或人工干预即可缓解过推荐并提升公平性。每个自对弈迭代包含SFT和DPO两步：将离线交互数据作为正样本，上一轮预测输出作为负样本。通过模型logits动态重加权DPO损失函数，自适应抑制偏见项。在多个真实数据集上的实验表明，SPRec有效提升了推荐准确率与公平性。代码已开源。
原文摘要 · Abstract (English)
Large language models (LLMs) have attracted significant attention in recommendation systems. Current work primarily applies supervised fine-tuning (SFT) to adapt the model for recommendation tasks. However, SFT on positive examples only limits the model's ability to align with user preference. To address this, researchers recently introduced Direct Preference Optimization (DPO), which explicitly aligns LLMs with user preferences using offline preference ranking data. However, we found that DPO inherently biases the model towards a few items, exacerbating the filter bubble issue and ultimately degrading user experience. In this paper, we propose SPRec, a novel self-play framework designed to mitigate over-recommendation and improve fairness without requiring additional data or manual intervention. In each self-play iteration, the model undergoes an SFT step followed by a DPO step, treating offline interaction data as positive samples and the predicted outputs from the previous iteration as negative samples. This effectively re-weights the DPO loss function using the model's logits, adaptively suppressing biased items. Extensive experiments on multiple real-world datasets demonstrate SPRec's effectiveness in enhancing recommendation accuracy and fairness. The implementation is available via https://github.com/RegionCh/SPRec
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。