通过降低专家间相关性，提升广告点击率预测精度
Enhancing CTR Prediction with De-correlated Expert Networks
- 引入跨专家去相关损失，主动降低各专家间依赖
- 实测显示专家去相关度提升，点击率预测性能显著增强
- 适合大规模推荐系统优化，尤其对MoE架构设计有指导意义
在广告系统中，建模特征交互对精准点击率（CTR）预测至关重要。近期研究采用专家混合（MoE）方法，通过集成多个特征交互专家来提升性能。现有策略如为每个专家学习独立嵌入表或使用异构专家结构，旨在实现专家去相关，但其有效性尚不明确。为此，本文提出去相关型MoE（D-MoE）框架，引入跨专家去相关损失以最小化专家间相关性，并提出新度量指标——跨专家相关性，用于定量评估去相关程度。基于该指标，发现不同去相关策略可相互兼容，逐步叠加能有效降低相关性并提升性能。大量实验验证了D-MoE的有效性及去相关原则。此外，在腾讯广告平台的在线A/B测试表明，相较于多嵌入MoE基线，D-MoE带来1.19%的总商品交易额（GMV）提升。
原文摘要 · Abstract (English)
Modeling feature interactions is essential for accurate click-through rate (CTR) prediction in advertising systems. Recent studies have adopted the Mixture-of-Experts (MoE) approach to improve performance by ensembling multiple feature interaction experts. These studies employ various strategies, such as learning independent embedding tables for each expert or utilizing heterogeneous expert architectures, to differentiate the experts, which we refer to expert de-correlation. However, it remains unclear whether these strategies effectively achieve de-correlated experts. To address this, we propose a De-Correlated MoE (D-MoE) framework, which introduces a Cross-Expert De-Correlation loss to minimize expert correlations.Additionally, we propose a novel metric, termed Cross-Expert Correlation, to quantitatively evaluate the expert de-correlation degree. Based on this metric, we identify a key finding for MoE framework design: different de-correlation strategies are mutually compatible, and progressively employing them leads to reduced correlation and enhanced performance. Extensive experiments have been conducted to validate the effectiveness of D-MoE and the de-correlation principle. Moreover, online A/B testing on Tencent's advertising platforms demonstrates that D-MoE achieves a significant 1.19% Gross Merchandise Volume (GMV) lift compared to the Multi-Embedding MoE baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。