用AI代理模拟电商页面测试,快速验证改版效果
SimGym: A Framework for A/B Test Simulation in E-Commerce with Traffic-Grounded VLM Agents

- 基于真实流量数据生成买家角色,让AI在浏览器中模拟购物
- 模拟结果与真实用户行为一致率达77%,实验时间从周缩短至一小时
- 适合想快速试错但又不想影响真实用户的电商平台
A/B测试仍是评估电商页面改动的黄金标准,但会分流真实流量、需数周才能获得显著结果,且可能降低用户体验。本文提出SimGym框架,利用视觉语言模型(VLM)代理在真实浏览器环境中模拟电商页面的A/B测试。该框架包含三个核心组件:(a) 基于生产点击流数据生成各店铺买家画像与意图的流量驱动型角色生成流程;(b) 结合多模态感知(视觉与浏览器结构化观察)、情景记忆与安全约束的实时浏览器代理架构,实现控制组与实验组页面间连贯的购物会话;(c) 比较模拟结果与真实用户行为变化的评估协议。我们在某大型电商平台对多个店铺和品类的视觉主导型界面主题变更进行了验证。实证结果显示,SimGym代理在加购行为变化方向上与真实流量观测结果达到77%的一致性,将实验周期从数周缩短至一小时内,可在不暴露真实用户的情况下实现快速迭代。
原文摘要 · Abstract (English)
A/B testing remains the gold standard for evaluating modifications to e-commerce storefronts, yet it diverts traffic, requires weeks to reach statistical significance, and risks degrading user experience. We present SimGym, a framework for simulating A/B tests on e-commerce storefronts using vision-language model (VLM) agents operating in a live browser. The framework comprises three key components: (a) a traffic-grounded persona generation pipeline that derives per-shop buyer archetypes and intents from production clickstream data; (b) a live-browser agent architecture that combines multimodal perception over visual and browser-structured observations with episodic memory and guardrails to conduct coherent shopping sessions across control and treatment storefronts; and (c) an evaluation protocol that compares simulated outcome shifts with observed shifts in real buyer behavior. We validate SimGym on A/B tests of visually driven UI theme changes from a major e-commerce platform across diverse storefronts and product categories. Empirical results show that SimGym agents achieve strong agreement with observed outcome shifts, attaining 77% directional alignment with add-to-cart shifts observed across interface variants in real-buyer traffic. It reduces experimental cycles from weeks to under an hour, enabling rapid experimentation without exposing real buyers to candidate variants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。