让手机自动化工具在新应用中快速适应，性能提升近15%。
CoAdapt-GUI: Joint Workflow Context and Policy Adaptation for Unseen GUI Applications
- 测试时联合优化操作流程与执行策略，利用自身交互数据自适应。
- 在未知应用上达到45.0%准确率，比基线高7.5个百分点。
- 适合需要快速适配新App的自动化工具研发者使用。
移动GUI代理在未见应用上仍显脆弱。本文研究在有限交互预算且无目标示范条件下，对新应用的泛化能力。提出CoAdapt-GUI，一种测试时自适应（TTA）框架，通过代理自身的目标应用回放和奖励信号，联合优化结构化工作流上下文与策略。工作流上下文保留可迁移的操作步骤、失败模式与验证规则，剔除源应用特有界面信息。该分离机制使可复用的工作流知识引导适应过程，而无需传递源界面状态。策略适应采用任务上下文匹配的组相对优化，对冻结的视觉-语言模型微调LoRA适配器。在两个未见应用评估中，CoAdapt-GUI在AndroidWorld-Generalization上达45.0%，优于报告的仅策略基线37.5%；在AndroidWorld Plus上从38.6%提升至52.9%。结果表明，受限于迁移的工作流上下文带来显著收益，联合策略适应进一步提升泛化性能。
原文摘要 · Abstract (English)
Mobile GUI agents remain brittle when deployed to applications absent from source training. We study novel-app generalization under a limited target interaction budget and without target demonstrations. We introduce CoAdapt-GUI, a test-time adaptation (TTA) framework that jointly adapts structured workflow context and policy from the agent's own target-app rollouts and rewards. The workflow context retains transferable procedures, failure modes, and verification rules while excluding app-bound source details. This separation allows reusable workflow knowledge to guide adaptation without transferring source-interface state. For policy adaptation, task-context-matched group-relative optimization updates a LoRA adapter on a frozen vision-language model. Across two unseen-app evaluations, CoAdapt-GUI reaches 45.0% on AndroidWorld-Generalization, compared with 37.5% for the reported Policy-Only TTA baseline, and raises AndroidWorld Plus performance from 38.6% to 52.9%. These results show that transfer-constrained workflow context provides substantial gains and that joint policy adaptation further improves held-out performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。