用自动分组与混合注意力，统一建模多场景推荐特征，提升点击和支付效果。
MTmixAtt: Integrating Mixture-of-Experts with Multi-Mix Attention for Large-Scale Recommendation
- 自动聚类异构特征为语义区块，替代人工分组。
- 在相同参数量下，点击率和支付转化率优于现有模型；10亿参数版本持续提升。
- 适合需要跨场景部署的工业级推荐系统，实测提升3.6%支付曝光量。
工业推荐系统依赖高质量排序模型，但传统流程仍依赖人工特征工程和场景特化架构，阻碍跨场景迁移与大规模部署。为此，我们提出MTmixAtt，一种融合门控专家（MoE）与多混合注意力的统一架构，适用于大规模推荐任务。其核心包含两个组件：AutoToken模块自动将异构特征聚类为语义一致的令牌，无需人工定义特征组；MTmixAttBlock模块通过可学习混合矩阵、共享稠密专家与场景感知稀疏专家，实现高效令牌交互，在同一框架中捕捉全局模式与场景特异性行为。在美团工业级TRec数据集上的大量实验表明，MTmixAtt持续优于基于Transformer的WuKong、HiFormer、MLP-Mixer与RankMixer等先进基线。在相近参数规模下，其点击率（CTR）与实际支付转化率（CTCVR）表现更优；扩展至MTmixAtt-1B后，性能进一步单调提升。大规模线上A/B测试验证了真实效果：在首页场景中，支付曝光量（Payment PV）提升+3.62%，实际支付交易额（Actual Payment GTV）提升+2.54%。整体上，MTmixAtt为跨场景任意异构特征建模提供了统一且可扩展的解决方案，显著改善用户体验与商业指标。
原文摘要 · Abstract (English)
Industrial recommender systems critically depend on high-quality ranking models. However, traditional pipelines still rely on manual feature engineering and scenario-specific architectures, which hinder cross-scenario transfer and large-scale deployment. To address these challenges, we propose \textbf{MTmixAtt}, a unified Mixture-of-Experts (MoE) architecture with Multi-Mix Attention, designed for large-scale recommendation tasks. MTmixAtt integrates two key components. The \textbf{AutoToken} module automatically clusters heterogeneous features into semantically coherent tokens, removing the need for human-defined feature groups. The \textbf{MTmixAttBlock} module enables efficient token interaction via a learnable mixing matrix, shared dense experts, and scenario-aware sparse experts, capturing both global patterns and scenario-specific behaviors within a single framework. Extensive experiments on the industrial TRec dataset from Meituan demonstrate that MTmixAtt consistently outperforms state-of-the-art baselines including Transformer-based models, WuKong, HiFormer, MLP-Mixer, and RankMixer. At comparable parameter scales, MTmixAtt achieves superior CTR and CTCVR metrics; scaling to MTmixAtt-1B yields further monotonic gains. Large-scale online A/B tests validate the real-world impact: in the \textit{Homepage} scenario, MTmixAtt increases Payment PV by \textbf{+3.62\%} and Actual Payment GTV by \textbf{+2.54\%}. Overall, MTmixAtt provides a unified and scalable solution for modeling arbitrary heterogeneous features across scenarios, significantly improving both user experience and commercial outcomes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。