将用户原始行为样本直接转为令牌,提升推荐模型信息利用率。
Sample Is Feature: Beyond Item-Level, Toward Sample-Level Tokens for Unified Large Recommender Models
- 把每个历史行为样本直接编码成令牌,保留完整上下文。
- 在真实工业数据上提升推荐效果,部署于外卖平台。
- 统一序列与非序列特征表示,释放模型全部潜力。
工业推荐模型的扩展遵循两条并行路径:样本信息扩展——通过更长更深的行为序列丰富单个训练样本的信息;以及模型容量扩展——在单一Transformer主干中统一序列建模与特征交互。然而,这两种路径仍存在结构性局限:第一,样本信息扩展方法仅将历史交互的一部分编码为序列令牌,导致多数原始样本上下文未被利用,无法建模样本级时变特征;第二,模型容量扩展方法受限于序列与非序列特征间的结构异质性,制约了模型表征能力的充分释放。为此,我们提出SIF(Sample Is Feature),将每个历史原始样本直接编码为序列令牌,最大化保留样本信息,同时解决序列与非序列特征间的异质性问题。SIF包含两个核心组件:样本分词器(Sample Tokenizer)通过层级分组自适应量化(HGAQ)将每个原始样本转化为令牌样本,高效融入全样本级上下文;SIF-Mixer通过令牌级与样本级混合机制,在同质化样本表示上实现深度特征交互,充分释放模型表征能力。大规模工业数据实验证明SIF有效性,已成功部署于某大型外卖平台。
原文摘要 · Abstract (English)
Scaling industrial recommender models has followed two parallel paradigms: \textbf{sample information scaling} -- enriching the information content of each training sample through deeper and longer behavior sequences -- and \textbf{model capacity scaling} -- unifying sequence modeling and feature interaction within a single Transformer backbone. However, these two paradigms still face two structural limitations. Firstly, sample information scaling methods encode only a subset of each historical interaction into the sequence token, leaving the majority of the original sample context unexploited and precluding the modeling of sample-level, time-varying features. Secondly, model capacity scaling methods are inherently constrained by the structural heterogeneity between sequential and non-sequential features, preventing the model from fully realizing its representational capacity. To address these issues, we propose \textbf{SIF} (\emph{Sample Is Feature}), which encodes each historical Raw Sample directly into the sequence token -- maximally preserving sample information while simultaneously resolving the heterogeneity between sequential and non-sequential features. SIF consists of two key components. The \textbf{Sample Tokenizer} quantizes each historical Raw Sample into a Token Sample via hierarchical group-adaptive quantization (HGAQ), enabling full sample-level context to be incorporated into the sequence efficiently. The \textbf{SIF-Mixer} then performs deep feature interaction over the homogeneous sample representations via token-level and sample-level mixing, fully unleashing the model's representational capacity. Extensive experiments on a large-scale industrial dataset validate SIF's effectiveness, and we have successfully deployed SIF on an industrial food delivery platform.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。