用流程引导训练降低推荐系统偏见，提升公平性与多样性。
Process-Supervised LLM Recommenders via Flow-guided Tuning
- 用生成流网络替代传统微调，通过逐标记奖励传递实现过程监督。
- 在多个数据集上显著降低流行度偏差，提升推荐公平性与多样性。
- 支持个性化偏好灵活调整，适合追求公平推荐的工业应用。
尽管大型语言模型（LLMs）通过监督微调（SFT）被广泛应用于推荐系统，但其基于似然最大化的目标会加剧流行度偏差，损害推荐的多样性和公平性。为此，我们提出流量引导微调推荐器（Flower），将SFT替换为生成流网络（GFlowNet）框架，通过标记级奖励传播实现过程监督。Flower的核心创新在于将项目级奖励分解为构成标记的奖励，使标记生成概率与奖励信号直接对齐。该机制实现了三大关键改进：（1）通过经验分布匹配缓解流行度偏差并提升公平性；（2）借助GFlowNet的比例采样保持多样性；（3）通过可调标记奖励灵活整合个性化偏好。实验表明，Flower在分布拟合能力上优于传统SFT，且在准确性、公平性和多样性方面均有显著优势，展现出改善基于LLM的推荐系统的潜力。代码已开源：https://github.com/MrPeach0301/Flower。
原文摘要 · Abstract (English)
While large language models (LLMs) are increasingly adapted for recommendation systems via supervised fine-tuning (SFT), this approach amplifies popularity bias due to its likelihood maximization objective, compromising recommendation diversity and fairness. To address this, we present Flow-guided fine-tuning recommender (Flower), which replaces SFT with a Generative Flow Network (GFlowNet) framework that enacts process supervision through token-level reward propagation. Flower's key innovation lies in decomposing item-level rewards into constituent token rewards, enabling direct alignment between token generation probabilities and their reward signals. This mechanism achieves three critical advancements: (1) popularity bias mitigation and fairness enhancement through empirical distribution matching, (2) preservation of diversity through GFlowNet's proportional sampling, and (3) flexible integration of personalized preferences via adaptable token rewards. Experiments demonstrate Flower's superior distribution-fitting capability and its significant advantages over traditional SFT in terms of accuracy, fairness, and diversity, highlighting its potential to improve LLM-based recommendation systems. The implementation is available via https://github.com/MrPeach0301/Flower
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。