让电脑操作智能体重复任务时提速8.5-13倍，靠的是记忆并重放之前成功执行的步骤。
PreAct: Computer-Using Agents that Get Faster on Repeated Tasks
- 首次成功后将操作流程编译为状态机程序，后续直接重放而非重新推理。
- 重放时每步检查屏幕是否匹配预期，不一致则立即交还控制权给智能体。
- 仅保留经独立验证成功的程序，避免错误累积导致任务失败。
计算机使用智能体通过屏幕点击和输入驱动真实软件，但每次任务都从头开始：重复任务时仍需重新读取界面、重新推理每一步操作，付出全部开销。我们提出 PreAct，使智能体在重复任务中变快。首次成功后，PreAct 将执行过程编译为小型状态机程序——状态检查屏幕，转移触发动作——后续运行直接重放该程序，速度提升 8.5-13 倍，且无需每步调用语言模型。重放非盲目：每步前检查屏幕是否符合程序预期，若不符则立即交还控制权给智能体。PreAct 采用相同标准决定是否保留程序：新编译的程序只有在从干净状态重跑时，经独立评估器确认完成任务，才会存入存储库，从而避免积累未完成的错误程序。在移动端、桌面端和网页端三个基准测试中，这一存储期检查机制使重复运行从劣化转向改进，平均多完成 1.75-2.6 个任务，方向一致；当无合适程序可用时，启用回退探索策略，使 PreAct 达到强基线记录与重放的水平。我们还发现：提示词措辞、运行时防护机制，以及选择程序的是否使用语言模型或纯嵌入检索器，均不影响结果。
原文摘要 · Abstract (English)
Computer-using agents drive real software through the screen -- clicking and typing -- but they solve every task from scratch: asked to repeat a task, an agent re-reads the screen, re-reasons every tap, and pays the full cost again. We present PreAct, which lets such an agent get faster on tasks it has done before. The first time it succeeds, PreAct compiles the run into a small state-machine program-states that check the screen, transitions that act-and on later runs replays it directly instead of invoking the agent 8.5-13x faster, with no per-step language-model calls. Replay is not blind: at each step PreAct checks that the screen matches what the program expects before acting, and hands control back to the agent the moment something is off. PreAct applies the same discipline when deciding what to keep: a freshly compiled program enters the store only if, re-run from a clean state, an independent evaluator confirms it solved the task-catching programs that replay to their last step yet leave the task undone. Across a mobile, a desktop, and a web benchmark, this store-time check separates repeated runs that improve from ones that degrade as faulty programs accumulate, worth 1.75-2.6 tasks per benchmark, the same direction on all three; a fallback that explores afresh when no program fits brings PreAct level with a strong record-and-replay baseline. We also report what did not matter: prompt wording, runtime guardrails, and whether a language model or a plain embedding retriever selects which program to reuse.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。