arXiv:2602.17410cs.IRcs.AI2026-02被引 1

用中间层自生成难负样本,提升大模型推荐效果

Improving LLM-based Recommendation with Self-Hard Negatives from Intermediate Layers

  • 从模型中间层提取细粒度难负样本作为监督信号
  • 两阶段优化框架提升负样本质量和模型判别力
  • 适合做大模型推荐系统、需改进负样本质量的研究者

大型语言模型在推荐系统中展现巨大潜力,通常通过监督微调(SFT)进行适配。后续研究引入偏好学习以融入负样本。然而,现有方法依赖序列级、离线生成的负样本,在负样本空间庞大的推荐任务中判别性与信息量不足。为此,我们提出ILRec,一种基于大模型推荐的新型偏好微调框架,利用中间层提取的自硬负样本信号增强偏好学习。具体而言,我们从中间层识别自硬负标记,作为细粒度负向监督,动态反映模型偏好学习过程。为有效整合这些信号,设计两阶段框架:跨层偏好优化与跨层偏好蒸馏,使模型同时区分有信息量的负样本并提升中间层负信号质量。此外,引入轻量协同过滤模型为负信号分配标记级奖励,缓解误罚假负样本风险。三个数据集上的大量实验表明,ILRec显著提升大模型推荐系统的性能。

原文摘要 · Abstract (English)

Large language models (LLMs) have shown great promise in recommender systems, where supervised fine-tuning (SFT) is commonly used for adaptation. Subsequent studies further introduce preference learning to incorporate negative samples into the training process. However, existing methods rely on sequence-level, offline-generated negatives, making them less discriminative and informative when adapting LLMs to recommendation tasks with large negative item spaces. To address these challenges, we propose ILRec, a novel preference fine-tuning framework for LLM-based recommendation, leveraging self-hard negative signals extracted from intermediate layers to improve preference learning. Specifically, we identify self-hard negative tokens from intermediate layers as fine-grained negative supervision that dynamically reflects the model's preference learning process. To effectively integrate these signals into training, we design a two-stage framework comprising cross-layer preference optimization and cross-layer preference distillation, enabling the model to jointly discriminate informative negatives and enhance the quality of negative signals from intermediate layers. In addition, we introduce a lightweight collaborative filtering model to assign token-level rewards for negative signals, mitigating the risk of over-penalizing false negatives. Extensive experiments on three datasets demonstrate ILRec's effectiveness in enhancing the performance of LLM-based recommender systems.

大模型推荐偏好学习负样本挖掘

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。