用自监督学习预测用户行为频次分布，提升广告系统建模精度
Abacus: Self-Supervised Event Counting-Aligned Distributional Pretraining for Sequential User Modeling
- 通过预测用户事件频次分布进行自监督预训练
- 下游任务收敛更快，最高提升6.1% AUC
- 适合需要精准用户行为建模的广告系统
在展示广告系统中，建模用户购买行为是实时竞价的关键挑战。问题源于正向用户事件稀疏和行为随机性，导致严重类别不平衡与事件时间不规则。现有预测系统依赖人工设计的“计数”特征，忽略用户意图的细粒度时间演化；同时，当前序列模型仅提取直接序列信号，遗漏了有用的事件计数统计。本文提出Abacus，一种用于展示广告的自监督预训练方法，通过预测用户事件的经验频次分布来增强深度序列模型。进一步设计混合目标函数，融合Abacus与序列学习目标，兼顾聚合统计的稳定性与序列建模的敏感性。在两个真实数据集上的实验表明，Abacus预训练优于现有方法，加速下游任务收敛；混合方法相较基线最高提升6.1% AUC。
原文摘要 · Abstract (English)
Modeling user purchase behavior is a critical challenge in display advertising systems, necessary for real-time bidding. The difficulty arises from the sparsity of positive user events and the stochasticity of user actions, leading to severe class imbalance and irregular event timing. Predictive systems usually rely on hand-crafted "counter" features, overlooking the fine-grained temporal evolution of user intent. Meanwhile, current sequential models extract direct sequential signal, missing useful event-counting statistics. We enhance deep sequential models with self-supervised pretraining strategies for display advertising. Especially, we introduce Abacus, a novel approach of predicting the empirical frequency distribution of user events. We further propose a hybrid objective unifying Abacus with sequential learning objectives, combining stability of aggregated statistics with the sequence modeling sensitivity. Experiments on two real-world datasets show that Abacus pretraining outperforms existing methods accelerating downstream task convergence, while hybrid approach yields up to +6.1% AUC compared to the baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。