为开放网页设计用户基础模型,提升实时竞价中无历史记录用户的推荐效果。
Building a User Foundation Model for the Open Web
- 用自监督学习建模碎片化浏览序列,保留时序信息。
- 在生产环境中实现点击率提升2.13%,每千次展示成本降低1.13%。
- 适合做广告推荐、用户行为建模的工业级应用研究者。
用户基础模型在电商和社交推荐中表现优异,但现有部署多假设用户身份稳定持久。开放网页实时竞价(RTB)环境下,用户身份跨会话断裂且不持久,浏览历史依赖隐私设置,导致大量流量无历史数据,可用记录多为短而离散的会话。传统方法仅用聚合计数与最近性分桶,忽略了序列结构。为此,我们提出一种用户基础模型,通过在用户浏览历史上进行自监督学习,证明其表征可提升多个下游任务。预训练采用掩码语言建模与序列级对比学习,再微调用于点击预测。利用大模型驱动的代码级优化(lifters),优化预训练流程。该编码器使生产级出价胜率模型提升+1.197% RIG,点击率排序器提升+1.354% RIG;7天线上A/B测试确认点击率+2.13%,每千次展示成本-1.13%(80%置信区间均不含零)。
原文摘要 · Abstract (English)
User foundation models have demonstrated strong results in e-commerce and social recommendation, but most industrial deployments assume environments where user identity is stable and persistent. Open-web real-time bidding (RTB) operates on a structurally different data distribution: user identity is fragmented and non-persistent across browsing sessions, and the availability of browsing history depends on user privacy choices. Consequently, a significant portion of traffic carries no historical data, and available records often consist of relatively short, disjointed sessions. As a result, historical signals in this domain are typically represented as aggregated counters and recency buckets, leaving the sequential structure unexploited. To address this limitation, we present a user foundation model that applies self-supervised learning on user browsing histories and show that the learned representation improves multiple downstream production tasks, demonstrating the viability of this approach on the open web. We pre-train a Transformer encoder with masked language modeling and a sequence-level contrastive objective, then fine-tune it on the click prediction task. We optimize the encoder's pre-training pipeline with an LLM-in-the-loop search over a curated catalog of reviewable, code-level edits (lifters), instantiating the LLM-as-optimizer paradigm in an industrial setting. The same encoder representation yields +1.197% RIG on the production bid win-rate model and +1.354% RIG on the production CTR ranker; a 7-day live A/B test confirms +2.13% CTR, -1.13% eCPC (80% CI excluding zero on both metrics).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。