arXiv:2602.14110cs.IR2026-02KDD被引 7

统一建模序列行为与特征交互,实现推荐系统高效共扩展。

MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders

  • 将序列建模与特征交互整合到同一骨干网络中
  • 在双平台测试中提升用户活跃天数与使用时长
  • 适合追求高精度与低延迟的工业级推荐系统

随着工业推荐系统进入规模驱动阶段,Transformer架构因可扩展性日益受到青睐。然而现有基于Transformer的推荐模型仍结构分散,序列建模与特征交互分属独立模块且参数独立,导致在有限算力下难以兼顾密集特征容量与序列长度,存在根本性的共扩展难题。本文提出MixFormer,一种专为推荐系统设计的统一Transformer架构,将序列行为与特征交互统一于单一主干网络中。通过统一参数化,实现密集容量与序列长度的有效共扩展,缓解解耦设计中的权衡问题。同时,集成架构促进序列与非序列表示的深层交互,使高阶特征语义直接指导序列聚合,增强模型表达能力。为保障工业实用性,进一步引入用户-物品解耦策略,显著降低冗余计算与推理延迟。大规模工业数据集实验表明,MixFormer在准确率与效率上均表现更优。在抖音及抖音极速版两个生产系统上的大规模在线A/B测试显示,用户参与度指标(活跃天数、应用内使用时长)持续提升。

原文摘要 · Abstract (English)

As industrial recommender systems enter a scaling-driven regime, Transformer architectures have become increasingly attractive for scaling models towards larger capacity and longer sequence. However, existing Transformer-based recommendation models remain structurally fragmented, where sequence modeling and feature interaction are implemented as separate modules with independent parameterization. Such designs introduce a fundamental co-scaling challenge, as model capacity must be suboptimally allocated between dense feature interaction and sequence modeling under a limited computational budget. In this work, we propose MixFormer, a unified Transformer-style architecture tailored for recommender systems, which jointly models sequential behaviors and feature interactions within a single backbone. Through a unified parameterization, MixFormer enables effective co-scaling across both dense capacity and sequence length, mitigating the trade-off observed in decoupled designs. Moreover, the integrated architecture facilitates deep interaction between sequential and non-sequential representations, allowing high-order feature semantics to directly inform sequence aggregation and enhancing overall expressiveness. To ensure industrial practicality, we further introduce a user-item decoupling strategy for efficiency optimizations that significantly reduce redundant computation and inference latency. Extensive experiments on large-scale industrial datasets demonstrate that MixFormer consistently exhibits superior accuracy and efficiency. Furthermore, large-scale online A/B tests on two production recommender systems, Douyin and Douyin Lite, show consistent improvements in user engagement metrics, including active days and in-app usage duration.

推荐系统Transformer共扩展工业部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。