让推荐策略自动迭代优化,减少人工干预。
A/B Agent: A Self-Evolving Agent for Strategy Iteration in Industrial A/B Testing

- 构建分层经验树,智能组织历史实验知识。
- 在线实验反馈驱动策略持续优化,提升GMV 4.829%。
- 适合需要高效迭代推荐策略的工业级平台使用。
工业推荐策略迭代高度依赖大规模A/B实验。传统方法需专家反复设计策略、配置实验、分析结果并调参,过程繁琐且耗时。历史实验中的知识常零散分布,难以通过人工系统复用。现有RAG代理虽能部分缓解负担,但多以扁平方式组织经验,忽略业务场景、推荐阶段、优化目标与实验上下文间的层级关系,导致检索不匹配、跨场景迁移有限,且无法通过连续A/B反馈实现策略与参数的自主优化。为此,我们提出A/B Agent,一种闭环式工业推荐策略优化代理。该框架包含三大核心组件:历史策略知识组织、自主目标感知策略生成、实验引导的策略自演化。它将历史策略构建成分层经验树,通过多路径Tree-RAG检索可迁移证据生成可执行策略,并基于在线A/B反馈持续分析,驱动策略自主调优并更新经验树以实现自我进化。大量离线与在线评估验证其有效性,在真实短视频电商推荐系统中实现GMV提升4.829%,同时所有护栏指标均保持正向增长。
原文摘要 · Abstract (English)
Industrial recommendation strategy iteration heavily relies on large-scale A/B experimentation. Traditional tuning requires experts to repeatedly design strategies, configure experiments, analyze results, and adjust parameters, making the process labor-intensive and time-consuming. Meanwhile, valuable knowledge from historical experiments is often fragmented, making systematic reuse difficult through manual expert effort alone. Existing RAG agents partially alleviate this burden by retrieving prior strategies, but typically organize experience in a flat manner, overlooking the hierarchical relationships among business scenarios, recommendation stages, optimization objectives, and experimental contexts. This often results in mismatched retrieval and limited cross-scenario transfer, while preventing agents from continuously refining strategies and parameters through sequential A/B feedback. % To address these limitations, we propose A/B Agent, a closed-loop A/B agent for industrial recommendation strategy optimization. The framework comprises three tightly coupled core components: Historical Strategy Knowledge Organization, Autonomous Target-Aware Strategy Generation, and Experiment-Guided Strategy Self-Evolution. It organizes historical strategies into a hierarchical experience tree, retrieves transferable evidence through multi-path Tree-RAG to generate executable strategies, and continuously analyzes online A/B feedback to guide autonomous tuning and update the experience tree for self-evolution. Extensive offline and online evaluations demonstrate its effectiveness, including a 4.829% improvement in GMV in a real-world short-video e-commerce recommendation system while maintaining positive gains across all guardrail metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。