首个面向电商风控的交互式评测基准,测试自动化网页代理的真实能力。
RiskWebWorld: A Realistic Interactive Benchmark for GUI Agents in E-commerce Risk Management

- 构建1513个真实风控任务,模拟网站对抗与环境劫持挑战。
- 顶尖通用模型成功率仅49.1%,专用模型近乎失败,暴露能力鸿沟。
- 支持强化学习训练,可提升开源模型性能16.2%,适配专业级自动化需求。
图形用户界面(GUI)代理在自动化网页任务中表现出强大能力,但现有交互式评测基准主要针对良性、可预测的消费环境,其在高风险、调查性领域如真实电商风控中的有效性尚未充分探索。为此,我们提出RiskWebWorld,首个高度真实的电商风控场景下GUI代理评测基准。该基准涵盖来自8个核心领域的1513个任务,源自实际风控流水线,真实还原了非合作网站与部分环境劫持带来的挑战。为支持可扩展评估与代理强化学习,我们进一步构建了符合Gymnasium标准的基础设施,实现策略规划与环境机制解耦。对多种模型的评估显示显著能力差距:顶级通用模型成功率达49.1%,而专用开源模型接近全面失败。这表明,在长周期专业任务中,基础模型规模目前比零样本界面理解更重要。我们还通过代理强化学习验证了基础设施可行性,使开源模型性能提升16.2%。这些结果使RiskWebWorld成为开发稳健数字工人的实用试验平台。
原文摘要 · Abstract (English)
Graphical User Interface (GUI) agents show strong capabilities for automating web tasks, but existing interactive benchmarks primarily target benign, predictable consumer environments. Their effectiveness in high-stakes, investigative domains such as authentic e-commerce risk management remains underexplored. To bridge this gap, we present RiskWebWorld, the first highly realistic interactive benchmark for evaluating GUI agents in e-commerce risk management. RiskWebWorld features 1,513 tasks sourced from production risk-control pipelines across 8 core domains, and captures the authentic challenges of risk operations on uncooperative websites, partially environmental hijackments. To support scalable evaluation and agentic reinforcement learning (RL), we further build a Gymnasium-compliant infrastructure that decouples policy planning from environment mechanics. Our evaluation across diverse models reveals a dramatic capability gap: top-tier generalist models achieve 49.1% success, while specialized open-weights GUI models lag at near-total failure. This highlights that foundation model scale currently matters more than zero-shot interface grounding in long-horizon professional tasks. We also demonstrate the viability of our infrastructure through agentic RL, which improves open-source models by 16.2%. These results position RiskWebWorld as a practical testbed for developing robust digital workers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。