用大模型自动把重复的电脑操作变成高效脚本
AutoRPA: Efficient GUI Automation through LLM-Driven Code Synthesis from Interactions
- 把大模型交互过程转成可复用的自动化代码
- 生成的脚本效率提升82%~96%，节省大量计算资源
- 适合需要反复做相似操作的技术人员或开发者
基于大语言模型（LLM）的智能体在多步骤图形界面交互中表现优异。然而，实际应用中常需重复执行相同任务，若每次均调用大模型推理（如ReAct范式），效率低下。传统机器人流程自动化（RPA）虽运行高效，但开发维护成本高。为此，我们提出AutoRPA框架，将ReAct式智能体的决策逻辑自动提炼为稳健的RPA函数。核心创新包括：(1) 翻译-构建流水线，翻译代理将硬编码的ReAct动作转化为软编码流程，构建代理通过多轨迹检索增强生成合成鲁棒的RPA函数；(2) 混合修复策略，在代码验证阶段结合RPA执行与ReAct回退进行迭代优化。在多个GUI环境中实验表明，AutoRPA生成的RPA函数能成功解决同类任务，同时减少82%至96%的令牌使用量，显著提升运行效率与可复用性。
原文摘要 · Abstract (English)
Large Language Model (LLM) based agents have demonstrated proficiency in multi-step interactions with graphical user interfaces (GUIs). While most research focuses on improving single-task performance, practical scenarios often involve repetitive GUI tasks for which invoking LLM reasoning repeatedly, i.e., the ReAct paradigm, is inefficient. Prior to LLMs, traditional Robotic Process Automation (RPA) offers runtime efficiency but demands significant manual effort to develop and maintain. To bridge this gap, we propose AutoRPA, a framework that automatically distills the decision logic of ReAct-style agents into robust RPA functions. AutoRPA introduces two core innovations: (1) A translator-builder pipeline, where a translator agent converts hard-coded ReAct actions into soft-coded procedures, and a builder agent synthesizes robust RPA functions via retrieval-augmented generation over multiple trajectories; (2) A hybrid repair strategy during code verification, combining RPA execution with ReAct-based fallback for iterative refinement. Experiments across multiple GUI environments demonstrate that RPA functions generated by AutoRPA successfully solve similar tasks while reducing token usage by 82% to 96%, significantly improving runtime efficiency and reusability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。