提出HeMix模型,提升工业推荐系统排序性能与扩展性。
HeMix: Scaling Industrial Ranking Models with Heterogeneous Token Mixing
- 用动态与固定查询统一建模用户上下文感知与不变意图。
- 设计HeteroMixer块实现线性成本的多粒度特征交互,提速降耗。
- 支持参数量从1亿到15亿平滑扩展,线上实验提升关键指标。
工业级推荐系统中,现有序列标记化无法同时捕捉来自异构行为源的上下文感知与上下文无关用户意图(挑战C1),且主流交互机制计算开销大、语义同质,难以满足严格在线延迟要求(挑战C2)。本文提出可扩展的排序模型HeMix,融合查询混合序列标记化与异构特征交互。为解决C1,引入查询混合兴趣提取模块,通过动态与固定查询并行建模全局与实时行为序列中的上下文感知与不变兴趣;为应对C2,设计HeteroMixer块,包含多头令牌融合、异构混合令牌交互与分组对齐重建,作为自注意力的高效替代方案,在线性计算成本下实现多粒度跨特征建模。重要的是,HeMix可独立扩展块深度与令牌维度,从约1亿到约15亿参数平滑扩展,无需重构架构即持续提升精度。工业数据实验表明,其在约1亿参数规模下相对DLRM基线提升+1.64% CTR-AUC,且所需GFLOPs少于最强竞争者。在高德地图APP上线的A/B测试显示,较生产基线提升+0.88% GMV、+2.74% PV_CTR、+0.84% UV_CVR。
原文摘要 · Abstract (English)
Scaling up ranking models for industrial recommender systems faces two critical challenges: (C1) existing sequence tokenization fails to jointly capture context-aware and context-invariant user intent from heterogeneous behavior sources, and (C2) prevailing interaction mechanisms are both computationally expensive and semantically homogeneous, limiting prediction quality under strict online latency constraints. We propose \textbf{HeMix}, a scalable ranking model that unifies query-mixed sequence tokenization with heterogeneous feature interaction. To address (C1), HeMix introduces a \textit{Query-Mixed Interest Extraction} module that employs dynamic and fixed queries to simultaneously model context-aware and context-invariant interests from global and real-time behavior sequences. To address (C2), we design the \textit{HeteroMixer} block, comprising Multi-Head Token Fusion, Heterogeneous Mixed-Token Interaction and Group-Aligned Reconstruction, as an efficient alternative to self-attention that enables multi-granularity cross-feature modeling at linear cost. Crucially, HeMix scales smoothly from ${\sim}100$M to ${\sim}1500$M parameters by independently expanding block depth and token dimension, yielding steady accuracy gains without architectural redesign. Experiments on industrial-scale data show that HeMix achieves $+1.64\%$ relative CTR-AUC over the DLRM baseline at the ${\sim}100$M scale while requiring fewer GFLOPs than the strongest competitor. Deployed on the AMAP APP, HeMix yields +0.88\% GMV, +2.74\% PV\_CTR and +0.84\% UV\_CVR over the production baseline in online A/B tests.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。