arXiv:2607.10016cs.IRcs.LG2026-07

将数值与嵌入特征转为软令牌,让大模型推荐系统更好利用非文本信号。

Tokenizing Numerical and Embedding Features for LLM RecSys

论文配图:Tokenizing Numerical and Embedding Features for LLM RecSys
图 1 · 摘自论文原文
  • 将连续数值和嵌入特征映射到大模型嵌入空间,生成软令牌。
  • 在三个Amazon数据集上,性能优于纯文本基线,提升显著。
  • 交互式融合模块比直接拼接更有效,适合多模态推荐场景。

大语言模型(LLMs)因其强大的序列建模和表征学习能力,正被广泛用于推荐系统。然而,多数基于LLM的推荐系统仅处理离散文本令牌,而实际推荐流程还依赖上游工程或预训练编码器生成的连续数值特征和密集嵌入特征。这种不匹配限制了模型对细粒度非文本信号的利用。本文提出一种软令牌融合框架,将数值与嵌入特征映射至LLM嵌入空间,使异构推荐信号可通过标准令牌接口输入。我们在共享参数的两塔检索模型中实现该框架,并引入基于交互的融合模块,在插入最终LLM输入前优化嵌入与数值软令牌。在三个Amazon推荐基准上的实验表明,软令牌融合显著优于基于LLM的基线,且交互式融合优于直接拼接。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly used as backbone architectures for recommender systems because of their strong sequence modeling and representation learning capabilities. However, most LLM-based recommenders operate primarily on discrete textual tokens, whereas practical recommendation pipelines also rely on continuous numerical features and dense embedding features produced by upstream feature engineering or pretrained encoders. This mismatch limits the ability of LLM-based models to exploit fine-grained non-textual signals. We propose a soft-token fusion framework that maps numerical and embedding features into the LLM embedding space, allowing heterogeneous recommendation signals to be consumed through the standard token interface. We instantiate the framework in a shared-parameter LLM-based two-tower retrieval model and introduce an interaction-based fusion module that refines embedding and numerical soft tokens before they are inserted into the final LLM input. Experiments on three Amazon recommendation benchmarks show that soft-token fusion improves retrieval performance over LLM-based baselines, and that interaction-based fusion is more effective than direct concatenation of heterogeneous soft tokens.

大模型推荐软令牌多模态融合两塔模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。