arXiv:2411.13700cs.IRcs.LG2024-11被引 13

用多个模型协作预测点击率,提升效果且更省资源。

A Collaborative Ensemble Framework for CTR Prediction

  • 多模型各自带独立嵌入表,通过协作学习捕捉不同特征交互模式。
  • 在多个数据集上优于单模型和当前最佳基线,小嵌入尺寸下仍表现优异。
  • 适合大规模推荐系统,尤其关注效率与性能平衡的工业场景。

近年来,基础模型的发展确立了规模扩展定律,推动了大规模推荐模型的研究。然而,单纯扩大推荐系统模型规模,并不能保证预期性能提升。本文提出一种新框架——协同集成训练网络(CETNet),利用多个具有独立嵌入表的模型,捕捉独特的特征交互模式。不同于简单的模型扩容,该方法强调多样性与协作,通过迭代式协同学习优化预测结果。为动态平衡各模型贡献,引入基于置信度的融合机制,使用通用softmax计算置信度,其由负熵决定:置信度高的模型对最终预测影响更大,同时受益于其他模型的互补优势。我们在三个公开数据集(AmazonElectronics、TaobaoAds、KuaiVideo)及Meta的工业级大数据集上验证了该框架,结果表明其显著优于单个模型与现有最优基线。进一步在Criteo和Avazu数据集上对比多嵌入范式,结果显示本方法在更小嵌入尺寸下仍能实现相当或更优性能,为点击率预测提供了一种高效可扩展的解决方案。

原文摘要 · Abstract (English)

Recent advances in foundation models have established scaling laws that enable the development of larger models to achieve enhanced performance, motivating extensive research into large-scale recommendation models. However, simply increasing the model size in recommendation systems, even with large amounts of data, does not always result in the expected performance improvements. In this paper, we propose a novel framework, Collaborative Ensemble Training Network (CETNet), to leverage multiple distinct models, each with its own embedding table, to capture unique feature interaction patterns. Unlike naive model scaling, our approach emphasizes diversity and collaboration through collaborative learning, where models iteratively refine their predictions. To dynamically balance contributions from each model, we introduce a confidence-based fusion mechanism using general softmax, where model confidence is computed via negation entropy. This design ensures that more confident models have a greater influence on the final prediction while benefiting from the complementary strengths of other models. We validate our framework on three public datasets (AmazonElectronics, TaobaoAds, and KuaiVideo) as well as a large-scale industrial dataset from Meta, demonstrating its superior performance over individual models and state-of-the-art baselines. Additionally, we conduct further experiments on the Criteo and Avazu datasets to compare our method with the multi-embedding paradigm. Our results show that our framework achieves comparable or better performance with smaller embedding sizes, offering a scalable and efficient solution for CTR prediction tasks.

CTR预测模型集成推荐系统高效学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。