统一特征交互与序列建模,提升推荐系统精度与效率。
UniDot: A Unified Network for Sequence Modeling and Feature Interaction in Large-scale Recommendation
- 用点积统一处理特征与序列,共享同一嵌入空间。
- 双路径并行计算,融合状态后预测点击转化率。
- 适合大规模工业推荐系统,推理延迟低。
工业级推荐系统长期依赖两类独立演化的模型:基于多领域用户/物品特征的特征交互模型,以及基于用户行为历史的序列模型。实际系统中二者耦合松散。为实现统一,本文提出UniDot,一种从因子分解机(FM)视角出发的新架构,用于点击后转化预测:嵌入向量的点积——支撑协同过滤并使推荐系统泛化至未见用户-物品对——与注意力机制中的查询-键打分本质相同,因此单一的令牌点积可同时支撑特征交互与序列建模。UniDot将非序列字段与多领域行为序列统一编码到共享令牌空间,采用单一宏块结构,其中令牌混合总线与序列检索总线并行运行,并通过MLP-Mixer在每层交换状态;同时,通过FM Highway直接将逐层点积交互传递至分类器。序列侧仅需一次前向传播嵌入,被所有下游任务共享,有效控制推理延迟。采用双优化器(Adagrad + Muon)、辅助转化延迟头及多路径互学习训练,UniDot在TAAC KDD Cup 2026工业赛道中获得亚军。
原文摘要 · Abstract (English)
Industrial recommenders rely on two model families that have evolved largely independently: feature-interaction models over multi-field user/item features, and sequential models over user-behavior histories. Production systems couple them only loosely. To unify the two, we present UniDot, a novel architecture for post-click conversion prediction built from the factorization-machine (FM) point of view: the embedding inner product---which powers collaborative filtering and lets a recommender generalize to unseen user--item pairs---is the same primitive as attention's query dot key scoring, so a single dot-product of tokens can underlie both feature interaction and sequence modeling. UniDot tokenizes non-sequential fields and multi-domain behavioral sequences into one shared token space and stacks a single macro-block in which a token-mixing bus and a sequence-retrieval bus (item tokens cross-attending the histories) run in parallel and exchange state each layer through an MLP-Mixer fusion, while an FM Highway carries explicit per-layer dot-product interactions around the residual stack directly to the classifier. The sequence side is embedded once per forward pass and shared by all consumers, bounding inference latency. Trained with a dual sparse/dense (Adagrad + Muon) optimizer, an auxiliary conversion-delay head, and multi-path mutual learning, UniDot finished as the runner-up on the Industrial track of the TAAC KDD Cup 2026.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。