一个Transformer统一建模用户行为与特征交互，提升推荐系统效率与效果。
OneTrans: Unified Feature Interaction and Sequence Modeling with One Transformer in Industrial Recommender
- 用统一分词器将行为序列和特征转为同一批令牌，共享参数增强信息融合。
- 在工业数据集上参数增长时表现稳定，线上测试用户GMV提升5.68%。
- 适合大规模推荐系统研发者，尤其关注高效可扩展架构的场景。
在推荐系统中，扩大特征交互模块（如Wukong、RankMixer）或用户行为序列模块（如LONGER）已取得显著成效。然而，这些方法通常独立演进，不仅阻碍双向信息交换，也难以实现统一优化与扩展。本文提出OneTrans，一种统一的Transformer骨干网络，同时完成用户行为序列建模与特征交互。OneTrans采用统一分词器，将序列与非序列属性转换为单一令牌序列。堆叠的OneTrans块在相似序列令牌间共享参数，同时为非序列令牌分配专属参数。通过因果注意力与跨请求键值缓存，实现中间表示的预计算与缓存，显著降低训练与推理阶段的计算开销。在工业级数据集上的实验表明，OneTrans能随参数增加高效扩展，持续超越强基线，在线上A/B测试中实现每用户GMV提升5.68%。
原文摘要 · Abstract (English)
In recommendation systems, scaling up feature-interaction modules (e.g., Wukong, RankMixer) or user-behavior sequence modules (e.g., LONGER) has achieved notable success. However, these efforts typically proceed on separate tracks, which not only hinders bidirectional information exchange but also prevents unified optimization and scaling. In this paper, we propose OneTrans, a unified Transformer backbone that simultaneously performs user-behavior sequence modeling and feature interaction. OneTrans employs a unified tokenizer to convert both sequential and non-sequential attributes into a single token sequence. The stacked OneTrans blocks share parameters across similar sequential tokens while assigning token-specific parameters to non-sequential tokens. Through causal attention and cross-request KV caching, OneTrans enables precomputation and caching of intermediate representations, significantly reducing computational costs during both training and inference. Experimental results on industrial-scale datasets demonstrate that OneTrans scales efficiently with increasing parameters, consistently outperforms strong baselines, and yields a 5.68% lift in per-user GMV in online A/B tests.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。