arXiv:2412.19064cs.AI2024-12被引 4

多通道竞价中动态分配预算,提升广告投放整体效果

Hierarchical Multi-agent Meta-Reinforcement Learning for Cross-channel Bidding

  • 分层多智能体强化学习,上层按渠道特征动态分配预算
  • 下层解耦策略减少离线训练偏差,提升跨渠道知识复用
  • 适用于有共享预算的跨平台广告投放优化场景

实时竞价(RTB)在在线广告生态系统中起关键作用。广告商通过策略性出价,在满足投资回报率(ROI)和每点击成本(CPC)等财务约束条件下优化广告影响力。传统方法主要针对固定预算,难以应对多渠道共享预算下的动态分配问题。本文提出一种分层多智能体强化学习框架,用于多通道竞价优化:上层采用基于CPC约束的扩散模型,根据各渠道特征与复杂依赖关系动态分配预算;下层使用状态-动作解耦的演员-评论家方法,缓解离线学习中因分布外动作导致的外推误差,并引入基于上下文的元渠道知识学习,提升策略的状态表征能力。在美团广告平台的大规模真实工业数据集上的实验表明,该方法达到当前最优性能。

原文摘要 · Abstract (English)

Real-time bidding (RTB) plays a pivotal role in online advertising ecosystems. Advertisers employ strategic bidding to optimize their advertising impact while adhering to various financial constraints, such as the return-on-investment (ROI) and cost-per-click (CPC). Primarily focusing on bidding with fixed budget constraints, traditional approaches cannot effectively manage the dynamic budget allocation problem where the goal is to achieve global optimization of bidding performance across multiple channels with a shared budget. In this paper, we propose a hierarchical multi-agent reinforcement learning framework for multi-channel bidding optimization. In this framework, the top-level strategy applies a CPC constrained diffusion model to dynamically allocate budgets among the channels according to their distinct features and complex interdependencies, while the bottom-level strategy adopts a state-action decoupled actor-critic method to address the problem of extrapolation errors in offline learning caused by out-of-distribution actions and a context-based meta-channel knowledge learning method to improve the state representation capability of the policy based on the shared knowledge among different channels. Comprehensive experiments conducted on a large scale real-world industrial dataset from the Meituan ad bidding platform demonstrate that our method achieves a state-of-the-art performance.

多智能体强化学习广告竞价预算分配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。