arXiv:2604.00590cs.IRcs.AI2026-04被引 10

统一推荐模型的缩放架构,提升训练效率和性能。

UniMixer: A Unified Architecture for Scaling Laws in Recommendation Systems

  • 将TokenMixer转为可学习参数结构,实现特征混合优化。
  • 支持任意头数与令牌数匹配,突破传统限制。
  • 适合追求高效缩放的推荐系统研发人员。

近年来,推荐模型的缩放规律受到越来越多关注,它描述了推荐系统性能与参数量/计算量之间的关系。目前主流的推荐模型缩放架构包括基于注意力、基于TokenMixer和基于因子分解机的方法,其设计哲学与结构存在本质差异。本文提出一种统一的推荐系统缩放架构——UniMixer,旨在提升缩放效率并建立统一的理论框架。通过将规则型TokenMixer转换为等价的参数化结构,构建了一个广义参数化特征混合模块,使令牌混合模式可在训练中优化学习。该设计打破了TokenMixer中头数必须等于令牌数的约束。同时,建立了统一的推荐系统缩放模块设计框架,连接了注意力、TokenMixer与因子分解机三类方法。为进一步提升缩放收益,设计了轻量级模块UniMixing-Lite,显著压缩参数与计算成本的同时大幅提高模型性能。大量离线与在线实验验证了UniMixer在缩放能力上的优越性。

原文摘要 · Abstract (English)

In recent years, the scaling laws of recommendation models have attracted increasing attention, which govern the relationship between performance and parameters/FLOPs of recommenders. Currently, there are three mainstream architectures for achieving scaling in recommendation models, namely attention-based, TokenMixer-based, and factorization-machine-based methods, which exhibit fundamental differences in both design philosophy and architectural structure. In this paper, we propose a unified scaling architecture for recommendation systems, namely \textbf{UniMixer}, to improve scaling efficiency and establish a unified theoretical framework that unifies the mainstream scaling blocks. By transforming the rule-based TokenMixer to an equivalent parameterized structure, we construct a generalized parameterized feature mixing module that allows the token mixing patterns to be optimized and learned during model training. Meanwhile, the generalized parameterized token mixing removes the constraint in TokenMixer that requires the number of heads to be equal to the number of tokens. Furthermore, we establish a unified scaling module design framework for recommender systems, which bridges the connections among attention-based, TokenMixer-based, and factorization-machine-based methods. To further boost scaling ROI, a lightweight UniMixing module is designed, \textbf{UniMixing-Lite}, which further compresses the model parameters and computational cost while significantly improve the model performance. The scaling curves are shown in the following figure. Extensive offline and online experiments are conducted to verify the superior scaling abilities of \textbf{UniMixer}.

推荐系统缩放定律统一架构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。