arXiv:2409.01605cs.IRcs.AI2024-09被引 2

用少量参数让大模型高效做推荐,提升精准度。

Laser: Parameter-Efficient LLM Bi-Tuning for Sequential Recommendation with Collaborative Information

  • 插入可训练虚拟标记,冻结大模型参数,仅微调前后缀
  • 融合用户-物品协同信息,推荐准确率显著超越现有方法
  • 轻量级M-Former捕捉不同类型用户特征,适合个性化推荐场景

序列推荐系统对理解用户历史行为、实现精准推荐至关重要。近期基于大语言模型(LLM)的方法通过编码物品语义推动了该领域发展,但通常需要大量参数调优且资源消耗高。此外,这些方法忽视了用户类型的多样性,导致推荐精度下降。本文提出一种参数高效的大型语言模型双向微调框架Laser,用于融合协同信息的序列推荐。具体而言,Bi-Tuning在输入序列的前缀和后缀插入可训练虚拟标记,同时冻结LLM参数,从而优化其用于序列推荐任务。前缀用于融合用户-物品协同信息,使LLM适配推荐任务;后缀则将LLM输出嵌入从语言空间转换至推荐空间,以支持后续物品预测。为在前缀中有效捕捉不同用户类型特征,引入M-Former——一种基于轻量级MoE的查询变压器,利用一组查询专家整合由冻结的基于ID的序列推荐模型编码的多样化用户特定协同信息,显著提升推荐准确性。在真实数据集上的广泛实验表明,Laser能以极低参数成本将大模型有效适配为推荐系统,显著优于当前最先进方法。

原文摘要 · Abstract (English)

Sequential recommender systems are essential for discerning user preferences from historical interactions and facilitating targeted recommendations. Recent innovations employing Large Language Models (LLMs) have advanced the field by encoding item semantics, yet they often necessitate substantial parameter tuning and are resource-demanding. Moreover, these works fails to consider the diverse characteristics of different types of users and thus diminishes the recommendation accuracy. In this paper, we propose a parameter-efficient Large Language Model Bi-Tuning framework for sequential recommendation with collaborative information (Laser). Specifically, Bi-Tuning works by inserting trainable virtual tokens at both the prefix and suffix of the input sequence and freezing the LLM parameters, thus optimizing the LLM for the sequential recommendation. In our Laser, the prefix is utilized to incorporate user-item collaborative information and adapt the LLM to the recommendation task, while the suffix converts the output embeddings of the LLM from the language space to the recommendation space for the follow-up item recommendation. Furthermore, to capture the characteristics of different types of users when integrating the collaborative information via the prefix, we introduce M-Former, a lightweight MoE-based querying transformer that uses a set of query experts to integrate diverse user-specific collaborative information encoded by frozen ID-based sequential recommender systems, significantly improving the accuracy of recommendations. Extensive experiments on real-world datasets demonstrate that Laser can parameter-efficiently adapt LLMs to effective recommender systems, significantly outperforming state-of-the-art methods.

序列推荐大模型参数效率协同过滤

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。