用户通过操作演示生成可复用的智能代理流程，更准确表达任务偏好。
ALLOY: Generating Reusable Agent Workflows from User Demonstration
- 用户通过实际操作演示代替文字指令，让系统学习任务流程。
- 实验显示其在复杂网页任务中更准确捕捉用户意图，成功率更高。
- 适合需要个性化流程、不想写复杂提示的普通用户使用。
大型语言模型（LLMs）使用户可通过自然语言将复杂任务委托给自主代理。然而基于提示的交互存在关键局限：用户难以准确描述程序性要求，尤其对于无唯一正确答案、依赖个人偏好的任务（如发布社交媒体内容或规划旅行）。此外，某一任务成功的提示未必能在类似任务中复用。我们提出 ALLOY，受经典人机交互中“演示编程”（Programming by Demonstration, PBD）理论启发，但扩展以增强基于 LLM 的网络代理的适应性。ALLOY 允许用户通过自然演示表达程序偏好，同时通过可视化工作流使流程透明且可编辑，支持跨任务变体的泛化。12 名参与者的实验表明，基于演示的方法在捕捉用户意图和程序偏好方面优于基于提示的代理和手动流程。研究还揭示了演示式交互如何补充传统提示方法。
原文摘要 · Abstract (English)
Large language models (LLMs) enable end-users to delegate complex tasks to autonomous agents through natural language. However, prompt-based interaction faces critical limitations: Users often struggle to specify procedural requirements for tasks, especially those that don't have a factually correct solution but instead rely on personal preferences, such as posting social media content or planning a trip. Additionally, a ''successful'' prompt for one task may not be reusable or generalizable across similar tasks. We present ALLOY, a system inspired by classical HCI theories on Programming by Demonstration (PBD), but extended to enhance adaptability in creating LLM-based web agents. ALLOY enables users to express procedural preferences through natural demonstrations rather than prompts, while making these procedures transparent and editable through visualized workflows that can be generalized across task variations. In a study with 12 participants, ALLOY's demonstration--based approach outperformed prompt-based agents and manual workflows in capturing user intent and procedural preferences in complex web tasks. Insights from the study also show how demonstration--based interaction complements the traditional prompt-based approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。