用智能代理强化学习迭代生成更精准的广告描述。
Interactor: Agentic RL oriented Iterative Creation for Ad Description Generation in Sponsored Search
- 设计多轮交互框架，通过奖励模型评估知识与一致性。
- 在真实工业数据集上显著优于现有方法，提升广告质量。
- 已上线主流搜索广告系统，兼顾收益与用户体验。
本文研究赞助搜索中自动生成信息丰富的广告描述。与通常为吸引点击优化的广告标题不同，广告描述文本更长，具备融入世界知识以回应用户搜索意图并呈现商品细节的优势。我们提出 Interactor，一种面向智能体强化学习的多轮迭代生成框架。生成模型作为策略，与包含多个生成式奖励模型的定制环境互动。策略初始生成后，定制的 GenRMs 从知识容量、落地页一致性等多维度评估质量，提供二值信号与推理反馈。策略据此迭代优化描述，实现持续改进。在工业数据集上的实验表明，Interactor 显著优于当前最优方法，生成更具知识性且忠实的广告描述。自 2026 年 5 月起，已在主流搜索广告系统上线，有效提升广告收入与用户体验。
原文摘要 · Abstract (English)
This paper focuses on automatically generating informative ad descriptions in sponsored search. Unlike ad titles which are usually optimized to attract user click feedbacks, ad descriptions have a longer text span and possess the potential of incorporating world knowledge to address user search intents while presenting the fine-grained selling points of the ads. We propose Interactor, a multi-turn iterative creation framework optimized with agentic RL for ad description generation. The generation model acts as a policy that interacts with a customized environment consisting of multiple generative reward models. Given initial generations by the policy, the customized GenRMs evaluate multi-dimensional qualities including knowledge capacity and landing page consistency, providing both binary signals and reasoning feedbacks. The policy then iteratively refines the descriptions based on such feedbacks to ensure continuous improvement. Experiments on industrial datasets show that the Interactor framework significantly outperforms state-of-the-art approaches in generating knowledge-rich and faithful ad descriptions. Since May 2026, it has been deployed online in a leading search ads system, contributing to both ad revenue and user experience.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。