用多个标识符增强推荐生成模型预训练数据多样性。
Pre-training Generative Recommender with Multi-Identifier Item Tokenization
- 每个物品用多个标识符表示,提升低频物品建模能力。
- 在三个数据集上优于传统与生成式推荐基线。
- 适合需要高推荐多样性和可扩展性的场景。
生成式推荐通过自回归生成物品标识符来推荐潜在物品。现有方法通常采用一对一映射策略,即每个物品仅由一个标识符表示,这导致低频物品语义建模不佳且标记序列数据多样性有限。为此,我们提出MTGRec,利用多标识符物品分词技术增强生成式推荐的预训练数据。该方法包含两项核心创新:多标识符物品分词和课程推荐预训练。多标识符物品分词中,以RQ-VAE作为分词器骨干,并将相邻训练轮次的模型检查点视为语义相关的分词器,使每个物品可关联多个标识符,从而将单一用户交互序列转化为多个不同数据组的标记序列。课程推荐预训练引入基于数据影响估计的课程学习策略,动态调整各数据组的采样概率。预训练后,使用单一分词器微调模型,确保推荐时物品标识的准确性。在三个公开基准数据集上的大量实验表明,MTGRec在效果和可扩展性方面显著优于传统及生成式推荐基线。
原文摘要 · Abstract (English)
Generative recommendation autoregressively generates item identifiers to recommend potential items. Existing methods typically adopt a one-to-one mapping strategy, where each item is represented by a single identifier. However, this scheme poses issues, such as suboptimal semantic modeling for low-frequency items and limited diversity in token sequence data. To overcome these limitations, we propose MTGRec, which leverages Multi-identifier item Tokenization to augment token sequence data for Generative Recommender pre-training. Our approach involves two key innovations: multi-identifier item tokenization and curriculum recommender pre-training. For multi-identifier item tokenization, we leverage the RQ-VAE as the tokenizer backbone and treat model checkpoints from adjacent training epochs as semantically relevant tokenizers. This allows each item to be associated with multiple identifiers, enabling a single user interaction sequence to be converted into several token sequences as different data groups. For curriculum recommender pre-training, we introduce a curriculum learning scheme guided by data influence estimation, dynamically adjusting the sampling probability of each data group during recommender pre-training. After pre-training, we fine-tune the model using a single tokenizer to ensure accurate item identification for recommendation. Extensive experiments on three public benchmark datasets demonstrate that MTGRec significantly outperforms both traditional and generative recommendation baselines in terms of effectiveness and scalability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。