提出MSL损失函数,让大模型推荐更准。
MSL: Not All Tokens Are What You Need for Tuning LLM as a Recommender
- 用掩码机制剔除错误负样本,优化训练信号
- 在4个数据集上平均提升NDCG@10达42.24%
- 适合做大模型推荐微调的研究者和工程师
大语言模型(LLM)因具备强大的理解能力与知识储备,正被越来越多地应用于推荐系统。然而,传统语言建模损失(LML)与推荐目标存在根本性偏差,且错误地将虚构商品描述视为负样本,引入误导性训练信号。为此,本文提出一种面向推荐微调的新型掩码软最大损失(MSL),通过识别并屏蔽可能导致虚构商品描述的无效令牌,有效避免错误负样本干扰,确保与推荐目标对齐,并获得理论保障。针对MSL可能存在的梯度消失问题,进一步提出自适应温度策略(ATS),无需大量超参数调优即可动态调节温度。在四个公开数据集上的大量实验验证了该方法的有效性,平均提升NDCG@10达42.24%。代码已开源。
原文摘要 · Abstract (English)
Large language models (LLMs), known for their comprehension capabilities and extensive knowledge, have been increasingly applied to recommendation systems (RS). Given the fundamental gap between the mechanism of LLMs and the requirement of RS, researchers have focused on fine-tuning LLMs with recommendation-specific data to enhance their performance. Language Modeling Loss (LML), originally designed for language generation tasks, is commonly adopted. However, we identify two critical limitations of LML: 1) it exhibits significant divergence from the recommendation objective; 2) it erroneously treats all fictitious item descriptions as negative samples, introducing misleading training signals. To address these limitations, we propose a novel Masked Softmax Loss (MSL) tailored for fine-tuning LLMs on recommendation. MSL improves LML by identifying and masking invalid tokens that could lead to fictitious item descriptions during loss computation. This strategy can effectively avoid the interference from erroneous negative signals and ensure well alignment with the recommendation objective supported by theoretical guarantees. During implementation, we identify a potential challenge related to gradient vanishing of MSL. To overcome this, we further introduce the temperature coefficient and propose an Adaptive Temperature Strategy (ATS) that adaptively adjusts the temperature without requiring extensive hyperparameter tuning. Extensive experiments conducted on four public datasets further validate the effectiveness of MSL, achieving an average improvement of 42.24% in NDCG@10. The code is available at https://github.com/WANGBohaO-jpg/MSL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。