让推荐模型理解物品间的层级结构,提升生成式推荐效果。
Trie-Aware Transformers for Generative Recommendation
- 引入两种位置编码,让Transformer捕捉物品的层级拓扑关系。
- 在四个真实数据集上平均提升8.83%的推荐性能。
- 无需调参、兼容主流生成式推荐模型,适合快速部署。
生成式推荐(GR)将下一个物品预测转化为基于分词的生成任务,而非传统的评分排序。现有方法采用两阶段流程:(i) 物品分词,将每个物品映射为层次化组织的离散标记序列;(ii) 自回归生成,根据用户历史交互的标记序列预测下一个物品的标记。尽管层次分词天然形成物品的前缀树(trie),但标准Transformer通常将标记线性处理,忽略其底层拓扑结构。为此,本文提出TrieRec,一种对前缀树敏感的生成式推荐方法,通过两种位置编码为Transformer注入结构归纳偏置。首先,使用【前缀树感知绝对位置编码】,将标记的局部结构上下文(如深度、祖先与后代)整合进标记表示。其次,采用【拓扑感知相对位置编码】,在自注意力中注入成对的结构关系,以捕捉由拓扑诱导的语义相关性。TrieRec具有模型无关性、高效性和无超参数特性。我们在三种代表性GR骨干网络中实现TrieRec,跨四个真实数据集平均性能提升8.83%。
原文摘要 · Abstract (English)
Generative recommendation (GR) aligns with advances in generative AI by casting next-item prediction as token-level generation rather than score-based ranking. Most GR methods adopt a two-stage pipeline: (i) \textit{item tokenization}, which maps each item to a sequence of discrete, hierarchically organized tokens; and (ii) \textit{autoregressive generation}, which predicts the next item's tokens conditioned on the tokens of user's interaction history. Although hierarchical tokenization induces a prefix tree (trie) over items, standard autoregressive modeling with conventional Transformers often flattens item tokens into a linear stream and overlooks the underlying topology. To address this, we propose TrieRec, a trie-aware generative recommendation method that augments Transformers with structural inductive biases via two positional encodings. First, a \textit{trie-aware absolute positional encoding} aggregates a token's (node's) local structural context (\eg depth, ancestors, and descendants) into the token representation. Second, a \textit{topology-aware relative positional encoding} injects pairwise structural relations into self-attention to capture topology-induced semantic relatedness. TrieRec is also model-agnostic, efficient, and hyperparameter-free. In our experiments, we implement TrieRec within three representative GR backbones, achieving notably improvements of 8.83\% on average across four real-world datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。