用生成模型+预测模型优化广告创意,实测效果提升超45%
Offline-to-Online Creative Optimization with Generative Models and Adaptive Testing
- 用历史实验数据训练预测模型,指导生成创意并筛选候选集
- 50臂实验中,最优生成创意比人工创作高45.1%互动率
- 适合广告、营销领域想高效试错的团队
广告创意优化正受限于评估而非生成。生成模型可产出大量合理创意,但可靠评估需依赖在线实验,且每次只能测试有限数量。我们研究如何利用历史A/B测试数据来生成和选择待测创意集。开发并部署了一种以性能为导向的离线-在线优化流程,使用预测模型作为推理时的评判标准。离线阶段,基于历史实验训练的预测模型对生成模型产生的变体进行排序与优化;最终生成的候选集在在线自适应实验中部署。在一项50臂实地实验中,最优生成创意的互动率比最优人工创作高出45.1%。另两项实验也呈现相同上尾效应,提升分别为46.7%和36.2%。尽管预测模型噪声较大无法直接识别最佳创意,但能有效引导生成模型产出强候选,可在自适应实验中高效评估。结果表明,生成式创意优化的设计原则是:用预测模型引导生成待测创意集,以是否包含高绩效候选为评价标准,再通过自适应实验选出优解,同时减少弱方案占用流量。
原文摘要 · Abstract (English)
Ad creative optimization is increasingly constrained by evaluation rather than generation. Generative models can produce many plausible creatives, but reliable evaluation requires online experiments, in which only a limited slate can be tested. We study how to use data from historical A/B tests to generate and select the candidates in that slate. We developed and deployed a performance-driven offline-to-online workflow that guides creative generation with a predictive model as an inference-time critic. In the offline phase, we use a predictive model trained on historical experiments to rank and refine variants created by a generative model. A final test slate is then deployed in an online adaptive experiment. In a 50-arm field experiment, we found that the best creative generated with this method yielded 45.1% higher engagement than the best human-authored creative. Two additional experiments showed the same upper-tail pattern, with lifts of 46.7% and 36.2%. We found that despite the predictive model being too noisy to directly identify the best creative offline, it effectively guides the generative model toward creating strong candidates that can be efficiently evaluated in an adaptive experiment. The results suggest a design principle for creative optimization with generative models: use predictive models to guide generation of a slate to test, judge the slate by whether it contains high-performing candidates at a feasible test size, and use adaptive experiments to select among candidates while limiting traffic lost to weak arms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。