对比大模型与文本嵌入模型在无训练冷启动推荐中的表现
Are Large Language Models Really Effective for Training-Free Cold-Start Recommendation?
- 在相同条件下对比LLM与TEM的推荐效果
- 文本嵌入模型在冷启动和热启动场景均优于大模型重排
- 挑战了大模型直接排序更优的普遍认知
推荐系统通常依赖大规模用户交互数据学习历史行为以实现精准预测,但在新服务上线或全新用户场景中常面临无训练数据的问题,传统方法无法适用。本文聚焦于无需训练的推荐任务,尤其关注目标用户完全无交互记录的极端情况——训练免费冷启动推荐(TFCSR)。尽管近期有研究探索大语言模型(LLMs)作为潜在解决方案,但尚无工作在相同条件下直接比较LLMs与文本嵌入模型(TEMs)的表现。本研究首次开展控制实验,系统评估两者在相同设置下的性能。结果表明,TEMs在冷启动及拥有丰富交互的热启动场景中均显著优于LLM重排器。这一发现挑战了‘大模型直接排序是唯一可行方案’的普遍信念,表明基于文本嵌入的方法为训练免费推荐提供了更强且更具可扩展性的基础。
原文摘要 · Abstract (English)
Recommender systems usually rely on large-scale interaction data to learn from users' past behaviors and make accurate predictions. However, real-world applications often face situations where no training data is available, such as when launching new services or handling entirely new users. In such cases, conventional approaches cannot be applied. This study focuses on training-free recommendation, where no task-specific training is performed, and particularly on \textit{training-free cold-start recommendation} (TFCSR), the more challenging case where the target user has no interactions. Large language models (LLMs) have recently been explored as a promising solution, and numerous studies have been proposed. As the ability of text embedding models (TEMs) increases, they are increasingly recognized as applicable to training-free recommendation, but no prior work has directly compared LLMs and TEMs under identical conditions. We present the first controlled experiments that systematically evaluate these two approaches in the same setting. The results show that TEMs outperform LLM rerankers, and this trend holds not only in cold-start settings but also in warm-start settings with rich interactions. These findings indicate that direct LLM ranking is not the only viable option, contrary to the commonly shared belief, and TEM-based approaches provide a stronger and more scalable basis for training-free recommendation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。