arXiv:2608.30449cs.LGcs.IR2026-08

提出PRIME模型,让点击率预测更精准且稳定。

PRIME: Mitigating Subgroup Optimization Competition in Shared CTR Top Networks with Plug-in Residual Input-Conditioned Mixture of Expert

  • 用输入条件的低秩专家修正主网络,保持原路径不变
  • 在Avazu和Criteo数据集上提升AUC达0.0022~0.0066
  • 适合追求高精度与低延迟的推荐系统开发者

点击率(CTR)模型在特征交互设计上差异较大,但其顶层网络通常为所有样本共享的单个多层感知机。因此,用户、物品和上下文的异构子群体更新相同参数,弱对齐的学习信号导致整体梯度成为竞争方向的折中。我们在Avazu数据集上评估了4种模型与4个语义字段,发现语义子群体的Top-NN梯度余弦相似度比随机匹配组低0.23–0.37。这促使我们设计输入条件专家,但直接替换原有全连接层会改变初始函数、共享模式与容量,掩盖增益来源。为此,我们提出PRIME(Plug-in Residual Input-conditioned Mixture of Experts),一种基于全连接锚点的低秩残差专家混合模型。它在训练初期通过零残差初始化精确匹配全连接基线,以输入依赖路由权重调用低秩专家进行实例化逻辑值修正;结合多袋聚合与指数移动平均负载偏差,稳定条件估计。我们在13种CTR架构与5组种子下评估PRIME在保留的Avazu与Criteo测试集上的表现。中位配对AUC提升0.0022与0.0066,对数损失分别降低0.0011与0.0081。在FiBiNET与DCNv2上,PRIME在全部十次种子对比中优于APG,同时参数更少、推理延迟更低。结果表明,功能保持的条件残差可增加输入相关容量,同时保留全连接路径及其优化稳定性。代码已开源。

原文摘要 · Abstract (English)

Click-through rate (CTR) models vary in feature-interaction design, yet their top networks usually remain a single multilayer perceptron shared by all examples. Heterogeneous user, item, and context subgroups therefore update the same parameters; weakly aligned learning signals make the aggregate gradient a compromise among competing directions. We study the competition on Avazu with 4 models and 4 semantic fields. Across all architectures, semantic subgroups show lower Top-NN gradient cosine similarity than random groups matched by sample size and label ratio, with reductions of 0.23-0.37. This competition motivates input-conditioned experts, but directly replacing an established Dense mapping changes its initial function, sharing pattern, and capacity, obscuring the source of gains. We introduce PRIME (Plug-in Residual Input-conditioned Mixture of Experts), a Dense-anchored mixture of low-rank residual experts. PRIME anchors the original prediction and uses zero-residual initialization to match the Dense baseline exactly at training onset. Input-dependent routing weights low-rank experts for example-specific logit corrections; multi-bag aggregation and EMA load biases stabilize conditional estimation. We evaluate PRIME on held-out Avazu and Criteo test sets across 13 CTR architectures and five paired seeds. Median paired AUC gains are +0.0022 and +0.0066, with LogLoss reductions of 0.0011 and 0.0081, respectively. On FiBiNET and DCNv2, PRIME outperforms APG in all ten seed-level AUC comparisons while using fewer parameters and lower inference latency on both backbones. These results show that function-preserving conditional residuals add input-dependent capacity while preserving the Dense path and its optimization stability. Code is available at https://github.com/YH-learning/PRIME.

CTR模型专家混合推荐系统低秩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。