arXiv:2602.11410cs.LG2026-02中稿 · KDD被引 1

用解码器架构提升广告点击率预测,解决上下文信号与线上一致性难题。

CADET: Context-Conditioned Ads CTR Prediction With a Decoder-Only Transformer

  • 采用上下文条件解码和多塔头设计,显式建模广告位置等后评分信号。
  • 在线测试中点击率提升11.04%,超越现有基线模型。
  • 适合工业级广告系统部署,兼顾高效训练与服务延迟优化。

点击率(CTR)预测是在线广告系统的核心。尽管深度学习推荐模型(DLRMs)长期主导该领域,但生成式推荐架构在内容推荐中展现出潜力。然而,将基于Transformer的模型应用于广告CTR预测仍面临挑战,包括处理后评分上下文信号、维持离线-线上一致性,以及应对工业级负载。本文提出CADET(Context-Conditioned Ads Decoder-Only Transformer),一个部署于LinkedIn的端到端解码器仅架构。其关键创新包括:(1)上下文条件解码架构与多塔预测头,显式建模广告位置等后评分信号,解决预估点击率与排序之间的鸡肋问题;(2)自门控注意力机制,通过自适应调节表示与交互层级的信息流,稳定训练;(3)基于时间戳的旋转位置编码(RoPE),捕捉从秒到月的时间尺度关系;(4)会话掩码策略,防止模型学习不可用会话内事件依赖,缓解训练-服务偏差;(5)生产工程优化,包括张量打包、序列分块和定制Flash Attention核,实现大规模高效训练与服务。在线A/B测试显示,CADET相较生产基线模型LiRank(DCNv2与序列编码器混合集成)提升11.04%点击率。系统已成功部署于LinkedIn广告平台,支撑首页推广内容的主要流量。

原文摘要 · Abstract (English)

Click-through rate (CTR) prediction is fundamental to online advertising systems. While Deep Learning Recommendation Models (DLRMs) with explicit feature interactions have long dominated this domain, recent advances in generative recommenders have shown promising results in content recommendation. However, adapting these transformer-based architectures to ads CTR prediction still presents unique challenges, including handling post-scoring contextual signals, maintaining offline-online consistency, and scaling to industrial workloads. We present CADET (Context-Conditioned Ads Decoder-Only Transformer), an end-to-end decoder-only transformer for ads CTR prediction deployed at LinkedIn. Our approach introduces several key innovations: (1) a context-conditioned decoding architecture with multi-tower prediction heads that explicitly model post-scoring signals such as ad position, resolving the chicken-and-egg problem between predicted CTR and ranking; (2) a self-gated attention mechanism that stabilizes training by adaptively regulating information flow at both representation and interaction levels; (3) a timestamp-based variant of Rotary Position Embedding (RoPE) that captures temporal relationships across timescales from seconds to months; (4) session masking strategies that prevent the model from learning dependencies on unavailable in-session events, addressing train-serve skew; and (5) production engineering techniques including tensor packing, sequence chunking, and custom Flash Attention kernels that enable efficient training and serving at scale. In online A/B testing, CADET achieves a 11.04\% CTR lift compared to the production LiRank baseline model, a hybrid ensemble of DCNv2 and sequential encoders. The system has been successfully deployed on LinkedIn's advertising platform, serving the main traffic for homefeed sponsored updates.

CTR预测Transformer广告系统在线学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。