用分词级嵌入初始化,让新用户新物品零交互也能推荐
Efficient Cold-Start Recommendation via BPE Token-Level Embedding Initialization with LLM
- 用BPE分词+预训练大模型生成细粒度词元嵌入
- 在无交互数据下仍达到更高召回率和排序效果
- 适合多语言、冷启动场景的轻量级推荐系统
冷启动问题是推荐系统的核心挑战,尤其在缺乏新用户或新商品历史交互数据时。传统基于内容或混合方法仅适用于元数据稀疏且模式浅显的场景。本文提出一种高效冷启动推荐策略:通过字节对编码(BPE)分词与预训练大语言模型(LLM)嵌入进行初始化,获得与BPE词汇表对齐的细粒度词元级向量,而非粗粒度句子嵌入。这些嵌入作为未见实体的密集语义先验,使无需交互历史即可实现即时推荐。该机制可类比协同过滤,在基准数据集上严苛冷启动假设下测试。实验表明,BPE-LLM方法在召回率@k、NDCG@k和命中率上均优于标准基线,且计算性能充足。进一步证明,子词感知嵌入具有更好泛化性与可解释性,尤其在多语言和稀疏输入环境下表现更优。本文证实,词元级语义初始化是一种轻量但有效的零样本推荐扩展方案。
原文摘要 · Abstract (English)
The cold-start issue is the challenge when we talk about recommender systems, especially in the case when we do not have the past interaction data of new users or new items. Content-based features or hybrid solutions are common as conventional solutions, but they can only work in a sparse metadata environment with shallow patterns. In this paper, the efficient cold-start recommendation strategy is presented, which is based on the sub word-level representations by applying Byte Pair Encoding (BPE) tokenization and pre-trained Large Language Model (LLM) embedding in the initialization procedure. We obtain fine-grained token-level vectors that are aligned with the BPE vocabulary as opposed to using coarse-grained sentence embeddings. Together, these token embeddings can be used as dense semantic priors on unseen entities, making immediate recommendation performance possible without user-item interaction history. Our mechanism can be compared to collaborative filtering systems and tested over benchmark datasets with stringent cold-start assumptions. Experimental findings show that the given BPE-LLM method achieves higher Recall@k, NDCG@k, and Hit Rate measurements compared to the standard baseline and displays the same capability of sufficient computational performance. Furthermore, we demonstrate that using subword-aware embeddings yields better generalizability and is more interpretable, especially within a multilingual and sparse input setting. The practical application of token-level semantic initialization as a lightweight, but nevertheless effective extension to modern recommender systems in the zero-shot setting is indicated within this work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。