用用户操作日志自动生成可复用的自动化流程，适应界面变化。
Log2Plan: An Adaptive GUI Automation Framework Integrated with Task Mining Approach
- 分两层规划：先从日志中提炼任务模式，再根据实时界面生成操作序列。
- 在200个真实任务上测试，长流程成功率超60%，执行更快。
- 适合需要个性化、长期稳定的GUI自动化场景。
GUI任务自动化能简化重复工作，但现有基于大模型或视觉语言模型的规划-执行代理存在泛化能力弱、延迟高、长程连贯性差的问题。其依赖单次推理或静态计划，在界面变化或复杂任务下表现脆弱。Log2Plan通过结合结构化双层规划框架与用户行为日志的任务挖掘方法，实现鲁棒且可适应的自动化。它将用户指令映射到结构化任务字典，构建高层级可复用的计划；利用用户行为日志挖掘个人化任务模式，提升个性化与重用性。高层计划通过实时解析GUI上下文转化为底层操作序列，确保跨不同界面的稳定执行。我们在200个真实任务上评估了Log2Plan，结果表明其任务成功率显著提升，执行时间更短。尤其在长程任务序列中仍保持超过60.0%的成功率，展现了在多步骤复杂工作流中的强鲁棒性。
原文摘要 · Abstract (English)
GUI task automation streamlines repetitive tasks, but existing LLM or VLM-based planner-executor agents suffer from brittle generalization, high latency, and limited long-horizon coherence. Their reliance on single-shot reasoning or static plans makes them fragile under UI changes or complex tasks. Log2Plan addresses these limitations by combining a structured two-level planning framework with a task mining approach over user behavior logs, enabling robust and adaptable GUI automation. Log2Plan constructs high-level plans by mapping user commands to a structured task dictionary, enabling consistent and generalizable automation. To support personalization and reuse, it employs a task mining approach from user behavior logs that identifies user-specific patterns. These high-level plans are then grounded into low-level action sequences by interpreting real-time GUI context, ensuring robust execution across varying interfaces. We evaluated Log2Plan on 200 real-world tasks, demonstrating significant improvements in task success rate and execution time. Notably, it maintains over 60.0% success rate even on long-horizon task sequences, highlighting its robustness in complex, multi-step workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。