提出真实世界移动代理评估框架，揭示其在复杂环境下的脆弱性。
AndroidReality: How Far Are Mobile Agents from the Real World?
- 从状态、转移、动作三方面构建接口扰动分类体系
- 在AndroidWorld上引入可控扰动，发现显著性能下降与四类错误模式
- 无需训练的测试时自省恢复机制可有效缓解各类失效
移动代理在干净的在线基准（如AndroidWorld）上表现良好，但在真实部署中常因环境变化和界面不完美而性能骤降。本文提出AndroidReality，一种基于扰动的移动代理鲁棒性评估与提升框架。通过马尔可夫决策过程（MDP）视角，将真实世界界面变异性系统归纳为状态、转移、动作三个维度的扰动。基于此分类，我们在AndroidWorld基础上构建了具备现实且可控扰动注入的扰动基准，实现移动代理鲁棒性的系统化评估。评估发现显著的鲁棒性差距及四类常见错误类型，据此提出无需训练的测试时自省恢复（TTIR）机制，在扰动与清洁场景下均能有效缓解故障。研究结果表明，鲁棒性是移动代理评估中缺失的关键维度，基准扰动是应力测试与暴露潜在弱点的有效工具。
原文摘要 · Abstract (English)
Mobile agents have achieved promising results on clean online benchmarks such as AndroidWorld, yet their performance often degrades sharply in real-world deployment due to environmental variations and imperfect interface conditions. In this work, we introduce AndroidReality, a perturbation-based framework for evaluating and improving the robustness of mobile agents. Through a Markov Decision Process (MDP) perspective, we organize real-world interface variability into a principled taxonomy of perturbations along three axes: state, transition, and action. Guided by this taxonomy, we build a perturbed mobile benchmark on top of AndroidWorld with realistic and controllable perturbation injections, enabling systematic robustness evaluation of mobile agents. Our evaluation reveals substantial robustness gaps and four recurring error categories, motivating a simple training-free Test-Time Introspective Recovery (TTIR) mechanism that mitigates these failures on both perturbed and clean settings. Together, these results position robustness as a missing dimension in mobile agent evaluation and establish benchmark perturbation as an effective tool for both stress testing and surfacing latent weaknesses of mobile agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。