主动发现用户网页操作模式，自动生成可执行自动化脚本。
Motif: Discovering and Automating Personal Web Workflows
- 通过被动观察浏览器行为，识别可自动化的重复操作模式。
- 8名用户参与测试，Motif发现的可自动化模式比用户主动尝试更多。
- 适合希望省去手动写脚本、提升日常效率的普通用户。
近年来，大语言模型（LLMs）和基于演示编程的研究使用户能够通过明确示范行为来创建自动化。然而，这些方法假设用户清楚哪些任务可自动化，且基于LLM代理的自动化成本较高。我们提出Motif系统，通过被动观察日常浏览器活动，发现可编程的重复交互模式，在检测到模式时向用户推荐，并在确认后生成可安装的程序。用户可用自然语言审查并优化生成的代码。我们在多日研究中评估Motif，将系统被动发现的结果与用户通过“感觉编码”（vibe coding）尝试构建的自动化进行对比。八名参与者中，Motif发现的可自动化模式数量超过用户自行识别的数量，多数匹配其常规操作且具有实用性。后续调查显示，大多数用户表示愿意继续使用Motif生成的程序。
原文摘要 · Abstract (English)
Recent advances in LLMs and existing work on programming by demonstration have made it possible for end users to create automations by explicitly demonstrating their behavior to LLMs. However, these approaches rely on the assumption that users know what to automate and what is capable of being automated. Additionally, automation via LLM agents is often expensive compared with programs. We introduce Motif, a system that passively observes everyday browser activity to discover recurring interaction patterns that are programmable, makes recommendations to users whenever a pattern is discovered and generate a program to install after user confirmation. Users can review, and refine the program using natural language. We evaluated Motif in a multi-day study, comparing its ambient discoveries against automations users attempted to build via ``vibe coding.'' With eight participants, Motif discovered more automatable patterns than users recognized. Most of them matched participants' routines and were useful. Follow-up surveys showed most would continue using Motif-generated programs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。