用语义子词替代固定原子，提升推荐系统对用户行为模式的捕捉能力
Rethinking Item Tokenization in Generative Recommenders: From Fixed Atoms to Semantic Subwords
- 将物品分解为可变长度的语义子词，减少编码器内冗余依赖
- 在三个数据集上相比基线模型点击率提升2.1%-3.7%
- 适合关注行为序列建模与生成式推荐的研究者
在生成式推荐系统中，物品通常被分割为固定长度的语义ID序列以进行自回归预测。然而，这种细粒度表示会导致用户上下文建模中的项内注意力过载：过多注意力被分配到低层级的项内依赖，而非高层级的项间行为转移。为此，我们提出语义子词标记法（SST），将历史物品表示为可变长度的语义子词，同时保持目标解码的固定长度。SST首先通过项级子词标记（IST）将稳定的相邻原子标记合并为紧凑的语义子词，从而降低编码器中的项内重构负担；随后引入行为诱导共现增强（BCA），注入粗粒度的语义前缀转移信号，引导释放出的建模能力聚焦于项间行为规律。在三个公开数据集和三种生成式推荐主干模型上的实验表明，SST显著优于固定长度及可迁移可变长度基线。代码已开源。
原文摘要 · Abstract (English)
In generative recommender systems, items are typically tokenized into fixed-length semantic ID sequences for autoregressive next-item prediction. However, for user-context modeling, this fine-grained representation triggers Intra-item Attention Overload: excessive attention is spent on low-level intra-item dependencies rather than high-level inter-item behavioral transitions. To address this, we propose Semantic Subword Tokenization (SST), which represents historical items as variable-length semantic subwords while preserving fixed-length target decoding. SST first applies Item-level Subword Tokenization (IST) to merge stable adjacent atom tokens into compact semantic subword tokens, thereby reducing intra-item reassembly in the encoder. It then introduces Behavior-induced Co-occurrence Augmentation (BCA) to inject coarse-grained semantic prefix transition signals, guiding the freed modeling capacity toward inter-item behavioral regularities. Extensive experiments on three public datasets and three generative recommender backbones show empirical improvements of SST over fixed-length and transferable variable-length SID baselines. Code is available at https://github.com/mxrcandy/Semantic-Subword-Tokenization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。