不依赖文本,用多个大模型的词嵌入做推荐,效果更稳更好。
Multi-LLM Token Filtering and Routing for Sequential Recommendation

- 通过用户引导过滤噪声词,只保留任务相关词嵌入。
- 融合多个大模型的语义空间,提升覆盖范围和表达能力。
- 无需额外文本或修改模型,适合想用大模型增强推荐的开发者。
大语言模型(LLM)在推荐系统中展现出潜力,因其能提供丰富的语义知识。现有方法通常依赖外部文本语料来对齐LLM与推荐系统,但本文重新审视一个更根本却未被充分探索的问题:推荐能否仅依靠单个LLM的词嵌入而无需文本输入?通过系统的实证研究,我们发现直接将单一LLM的词嵌入注入序列推荐器会导致性能不稳定或提升有限,原因在于语义错位、任务适配不足及单个LLM覆盖范围受限。为此,我们提出MLTFR框架——多大模型词嵌入筛选与路由,用于无语料序列推荐。该框架采用交互引导的知识融合范式,通过用户引导的词嵌入筛选机制抑制无关词汇信号;同时引入基于费舍尔加权语义共识的专家混合架构,整合多个LLM的词嵌入空间,平衡异构专家并防止训练中主导现象。通过联合筛选有效词与聚合互补语义知识,MLTFR实现了无需文本输入或主干网络修改的稳定高效利用。大量实验表明,MLTFR持续优于现有顶尖序列推荐基线及对齐方法。代码已开源:https://github.com/ccwwhhh/MLTFR。
原文摘要 · Abstract (English)
Large language models (LLMs) have recently shown promise in recommendation by providing rich semantic knowledge. While most existing approaches rely on external textual corpora to align LLMs with recommender systems, we revisit a more fundamental yet underexplored question: Can recommendation benefit from LLM token embeddings alone without textual input? Through a systematic empirical study, we show that directly injecting token embeddings from a single LLM into sequential recommenders leads to unstable or limited gains, due to semantic misalignment, insufficient task adaptation, and the restricted coverage of individual LLMs. To address these challenges, we propose MLTFR, a Multi-LLM Token Filtering and Routing framework for corpus-free sequential recommendation. MLTFR follows an interaction-guided LLM knowledge integration paradigm, where task-relevant token embeddings are selected via user-guided token filtering to suppress noisy and irrelevant vocabulary signals. To overcome the limitations of single-LLM representations, MLTFR integrates multiple LLM token spaces through a Mixture-of-Experts architecture, with a Fisher-weighted semantic consensus expert to balance heterogeneous experts and prevent domination during training. By jointly filtering informative tokens and aggregating complementary semantic knowledge across multiple LLMs, MLTFR enables stable and effective utilization of LLM token embeddings without textual inputs or backbone modification. Extensive experiments demonstrate that MLTFR consistently outperforms state-of-the-art sequential recommendation baselines and existing alignment methods. Our code is available at: https://github.com/ccwwhhh/MLTFR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。