测试大模型能否理解用户物品互动数据，发现加引导提示后表现超传统方法。
Do LLMs Understand Collaborative Signals? Diagnosis and Repair
- 用检索增强生成技术让大模型读取用户-物品交互矩阵
- 提供清晰信息并引导推理时，大模型性能超过矩阵分解模型
- 信息越多、提示越明确，大模型表现越好，适合推荐系统研究者
用户-物品交互产生的协同信息是成功推荐系统的核心信号。近期研究尝试将此类知识引入基于大语言模型的推荐方法（LLMRec）以提升性能，但尚缺乏对大模型是否能有效推理协同信息的根本性分析。本文评估大模型在推荐任务中推理协同信息的能力，并与传统矩阵分解（MF）模型对比。提出一种简单有效的改进方法：通过四种不同提示策略，利用检索增强生成（RAG）对用户-物品交互矩阵进行增强。实验表明，当提供相关且结构清晰的信息并引导大模型基于其推理时，大模型性能始终优于MF模型；几乎在所有情况下，提供的信息越多，大模型表现越好。
原文摘要 · Abstract (English)
Collaborative information from user-item interactions is a fundamental source of signal in successful recommender systems. Recently, researchers have attempted to incorporate this knowledge into large language model-based recommender approaches (LLMRec) to enhance their performance. However, there has been little fundamental analysis of whether LLMs can effectively reason over collaborative information. In this paper, we analyze the ability of LLMs to reason about collaborative information in recommendation tasks, comparing their performance to traditional matrix factorization (MF) models. We propose a simple and effective method to improve LLMs' reasoning capabilities using retrieval-augmented generation (RAG) over the user-item interaction matrix with four different prompting strategies. Our results show that the LLM outperforms the MF model whenever we provide relevant information in a clear and easy-to-follow format, and prompt the LLM to reason based on it. We observe that with this strategy, in almost all cases, the more information we provide, the better the LLM performs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。