测试安卓界面智能体在运行时异常下的鲁棒性，发现多数模型严重失效。
Are Android GUI Agents Robust Against Runtime Anomalies? AnTrap: Evaluating Agents in Dynamic Adversarial Environments
- 构建四层十类动态异常分类体系，模拟真实运行干扰。
- 16个主流模型均显著降效，最强模型性能仍暴跌40%以上。
- 揭示训练环境陷阱无法解决深层状态死锁等根本问题。
GUI智能体在安卓设备部署时常遭遇动态异常，如意外弹窗、操作误用等，但现有基准缺乏对智能体鲁棒性的系统评估。本文提出AnTrap，一个全面的基准，通过向智能体执行轨迹注入动态扰动进行评估。我们构建了一个四层（状态、思考、动作、回合）分类体系，包含十类细粒度子类别，并设计了保持任务可解性的同时引入现实对抗条件的构造流程。对16个领先GUI模型的评估显示，普遍对动态异常脆弱，即使最强模型性能也显著下降。进一步在原始与对抗环境中进行GRPO训练，验证基准有效性，分离出可由环境学习解决的异常与推理瓶颈导致的问题。结果表明，单步陷阱（状态与动作层）可通过对抗强化学习缓解，但深层上下文陷阱（如状态死锁）暴露了模型内在局限，仅靠含陷阱环境训练无法解决。
原文摘要 · Abstract (English)
GUI agents often encounter dynamic anomalies when deployed on Android devices, from unexpected pop-ups to action misuse, yet existing benchmarks lack systematic evaluation of agent robustness against runtime anomalies. We introduce AnTrap, a comprehensive benchmark that injects dynamic perturbations into agent execution trajectories. We propose a taxonomy organizing real-world anomalies into four layers (State, Thinking, Action and Round) with ten fine-grained subcategories, and develop a construction pipeline that preserves task solvability while introducing realistic adversarial conditions. Evaluating 16 leading GUI models, we reveal universal vulnerability to dynamic anomalies, with even the strongest models suffering significant performance degradation. Furthermore, we conduct GRPO training in both original and adversarial environments to validate our benchmark, separating environment-learnable anomalies from reasoning-bottlenecked ones. Our findings show that while single-step traps at state and action layers are largely addressable through adversarial reinforcement learning, deep contextual traps, like state deadlock, expose intrinsic limitations that cannot be resolved by training in environments with traps alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。