通过自对弈机制,让大模型推荐系统自我纠正不公平问题。
UFO: Unfair-to-Fair Evolving Mitigates Unfairness in LLM-based Recommender Systems via Self-Play Fine-tuning
- 设计双角色自对弈框架:裁判识别偏见,修正者调整模型。
- 在多个数据集上同时降低不公平性并提升推荐效果。
- 适合关注模型公平性与性能平衡的研究者和工程师。
基于大语言模型的推荐系统(LRS)虽通过预训练与监督微调(SFT)实现了优异推荐性能,但引入了物品侧不公平问题。现有研究多归因于SFT阶段缺乏公平性约束,采用重加权或重排序方法缓解。本文发现,不公平性不仅源于SFT,更根植于预训练阶段,且在SFT中被进一步放大。这揭示了现有方法未能触及根本原因。此外,这些方法往往难以兼顾推荐性能。为此,我们提出无偏演化框架UFO,采用自对弈机制,将公平性缓解建模为两玩家博弈。UFO交替扮演“裁判”与“修正者”角色:裁判从预训练与SFT中识别不公平,修正者则在保持推荐性能前提下调整模型。通过角色间迭代优化,UFO可完全消除不公平性。大量实验表明,UFO不仅能有效缓解不公平性,还提升了推荐性能。
原文摘要 · Abstract (English)
Large language model-based Recommender Systems (LRSs) have demonstrated superior recommendation performance by integrating pre-training with Supervised Fine-Tuning (SFT). However, this approach introduces item-side unfairness. Existing studies primarily attribute this issue to the absence of fairness constraints during SFT and attempt to mitigate unfairness via re-weighting and re-ranking methods. In this paper, we find that unfairness arises not only from SFT but also from pre-training, where inherent biases are further amplified during SFT. This finding underscores the failure of current methods to address the root causes of unfairness. Moreover, current methods struggle to preserve satisfactory recommendation performance. To tackle these issues, we propose an Unfair-to-Fair evOlving (UFO) framework using a self-play mechanism, formulating unfairness mitigation as a two-player game. UFO alternates between two player roles: the \textit{judger}, which identifies unfairness from both pre-training and SFT, and the \textit{corrector}, which adjusts the LRS to address identified unfairness while preserving recommendation performance. Iterative optimization between these roles enables UFO to completely resolve unfairness. Extensive experiments demonstrate that UFO effectively mitigates unfairness while improving recommendation performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。