提出EST模型,实现点击率预测的统一建模与高效扩展。
EST: Towards Efficient Scaling Laws in Click-Through Rate Prediction via Unified Modeling
- 将用户行为全量输入统一处理,避免信息丢失。
- 在淘宝广告平台实现CTR提升1.22%,收益增长3.27%。
- 适合需要高效可扩展点击率预测的工业场景。
高效扩展工业级点击率(CTR)预测近年来受到广泛关注。现有方法通常采用早期聚合用户行为以保持效率,但这种非统一或部分统一建模会因丢弃细粒度的标记级信号而产生信息瓶颈,阻碍规模化收益。本文重新审视了CTR预测与大语言模型(LLMs)的根本差异,识别出两个关键特性:行为与非行为特征间的信息密度不对称性,以及内容丰富信号的模态特异性先验。为此,我们提出高效可扩展变换器(EST),通过在单一序列中处理所有原始输入,实现完全统一建模。EST集成两个模块:轻量级交叉注意力(LCA),通过剪枝冗余自交互,聚焦高影响的跨特征依赖;内容稀疏注意力(CSA),利用内容相似性动态选择高信号行为。大量实验表明,EST展现出稳定且高效的幂律扩展关系,支持模型规模下的可预测性能提升。在淘宝展示广告平台部署后,显著优于生产基线,带来3.27%的每千次展示收入(RPM)提升和1.22%的CTR增长,为可扩展工业级CTR模型提供了实用路径。
原文摘要 · Abstract (English)
Efficiently scaling industrial Click-Through Rate (CTR) prediction has recently attracted significant research attention. Existing approaches typically employ early aggregation of user behaviors to maintain efficiency. However, such non-unified or partially unified modeling creates an information bottleneck by discarding fine-grained, token-level signals essential for unlocking scaling gains. In this work, we revisit the fundamental distinctions between CTR prediction and Large Language Models (LLMs), identifying two critical properties: the asymmetry in information density between behavioral and non-behavioral features, and the modality-specific priors of content-rich signals. Accordingly, we propose the Efficiently Scalable Transformer (EST), which achieves fully unified modeling by processing all raw inputs in a single sequence without lossy aggregation. EST integrates two modules: Lightweight Cross-Attention (LCA), which prunes redundant self-interactions to focus on high-impact cross-feature dependencies, and Content Sparse Attention (CSA), which utilizes content similarity to dynamically select high-signal behaviors. Extensive experiments show that EST exhibits a stable and efficient power-law scaling relationship, enabling predictable performance gains with model scale. Deployed on Taobao's display advertising platform, EST significantly outperforms production baselines, delivering a 3.27\% RPM (Revenue Per Mile) increase and a 1.22\% CTR lift, establishing a practical pathway for scalable industrial CTR prediction models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。