arXiv:2510.10978cs.IR2025-10中稿 · WWW2026被引 8

提出新方法减轻大模型推荐中的上下文偏差,提升准确率与公平性。

Does LLM Focus on the Right Words? Mitigating Context Bias in LLM-based Recommenders

  • 通过分组分布鲁棒优化,引导模型关注用户行为数据而非辅助提示词。
  • 在三个数据集上平均提升NDCG@10达24.29%,显著改善推荐效果。
  • 适合关注推荐系统公平性与模型可解释性的研究者和开发者。

大型语言模型因其广泛的开放域知识和语义推理能力,被越来越多地集成到推荐系统中。然而,其预训练目标与推荐任务需求之间存在显著差距。为缩小这一差距,通常在专门构建的推荐数据集上进行监督微调(SFT),以增强预测能力。尽管成效显著,但SFT存在一个关键缺陷:引发上下文偏差,即模型过度依赖任务描述、前缀生成等辅助标记,而忽视了编码用户特定偏好的核心交互标记。这种偏差不仅降低推荐准确性,还带来不公平问题。为此,我们提出基于分组分布鲁棒优化的微调方法(GDRT),强制模型在与辅助标记相关性不同的标记组间保持一致性能表现。通过自适应加权低表现组(通常与辅助标记弱相关),GDRT将模型注意力从表面辅助线索转向有信息量的用户交互标记,从而缓解上下文偏差。在三个公开数据集上的大量实验表明,GDRT有效缓解上下文偏差,在推荐准确率上实现显著提升(平均NDCG@10提升24.29%),并大幅增强推荐公平性。代码已开源。

原文摘要 · Abstract (English)

Large language models (LLMs), owing to their extensive open-domain knowledge and semantic reasoning capabilities, have been increasingly integrated into recommender systems (RS). However, a substantial gap remains between the pre-training objectives of LLMs and the specific requirements of recommendation tasks. To address this gap, supervised fine-tuning (SFT) is commonly performed on specially curated recommendation datasets to further enhance their predictive ability. Despite its success, SFT exhibits a critical limitation: it induces Context Bias, whereby the model over-relies on auxiliary tokens, such as task descriptions and prefix-generated tokens, while underutilizing core user interaction tokens that encode user-specific preferences. This bias not only undermines recommendation accuracy but also raises unfairness concerns. To address this issue, we propose Group Distributionally Robust Optimization-based Tuning (GDRT), a novel fine-tuning paradigm that enforces consistent model performance across token groups with varying degrees of relevance to auxiliary tokens. By adaptively upweighting underperforming groups, typically those weakly correlated with auxiliary tokens, GDRT shifts the model's attention from superficial auxiliary cues to informative user interaction tokens, thereby mitigating context bias. Extensive experiments conducted on three public datasets demonstrate that GDRT effectively mitigates context bias, yielding substantial improvements in recommendation accuracy (with an average NDCG@10 gain of 24.29%) and significantly enhancing recommendation fairness. The code is available at https://github.com/WANGBohaO-jpg/GDRT.

推荐系统大模型偏差缓解公平性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。