让广告代理自我进化,更懂何时该推、推什么。
AdsWorldEngine: A Self-Evolving Conversational Advertising Agent through Orchestrator and Tool Coevolution

- 用智能调度器和工具协同训练,实现广告决策与工具的持续优化。
- 离线提升广告多样性60%、相关性80%,线上点击收益增22%。
- 适合做对话广告系统研发或想提升广告精准度的团队。
对话式广告旨在多轮交互中提供有用广告。与传统基于查询的广告不同,对话广告需从当前用户问题、助手回复及对话历史中推断潜在商业意图,并判断广告是否有益而非打扰。我们提出 AdsWorldEngine,一种用于对话广告的智能体框架:通过机会门判断是否展示广告,调度器生成商业意图、调用广告工具并构建前3名广告候选集,评估器对投放广告打分以支持离线优化。核心创新是迭代式智能体-工具训练流程:先用监督微调与智能体强化学习训练调度器,再利用高/低奖励轨迹构建偏好数据训练工具,形成自我改进闭环。此外引入标签引导的判断建模,基于人类标注训练判断模型,融合思维路径,通过反思过滤不一致理由,并使用成本敏感的GRPO变体保持非对称奖励差距。离线实验显示,相比现有生产系统,广告多样性提升60%,相关性提升80%;在线A/B测试中,每千次展示收益(RPM)提升22%,广告覆盖率提升74%。
原文摘要 · Abstract (English)
Conversational advertising aims to deliver useful ads within multi-turn assistant interactions. Unlike conventional query-based advertising, where the user's intent is often expressed in a short standalone query, conversational ads must infer latent commercial intent from the current user query, the assistant response, and dialogue history while also deciding whether an ad would be helpful rather than intrusive. We propose AdsWorldEngine, an agentic framework for conversational advertising. AdsWorldEngine uses an Opportunity Gate to determine whether ads should be shown, an Orchestrator to generate commercial intents, call advertising tools, and construct a top-3 ad slate, and an Evaluator to score delivered ads for offline optimization. The central contribution is an iterative actor-tool training procedure: we first train the Orchestrator with supervised fine-tuning and agentic reinforcement learning, then use high- and low-reward rollouts to construct preference data to train tools. This creates a self-improving loop in which the system learns not only how to use advertising tools, but also how to improve them from rewarded behavior. To support subjective production decisions, we introduce label grounded judgment modeling, which trains judgment models from human labels collected under explicit guidelines. It enriches labels with thinking traces, filters inconsistent rationales through reflection, and further optimizes binary judgments with a cost sensitive GRPO variant that preserves asymmetric reward gaps. Offline, AdsWorldEngine improves diversity by 60% and relevance by 80% over the current production ad delivery system. In an online A/B test, it increases RPM by 22% and ads coverage by 74%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。