用自然语言写操作流程，让AI agent更懂专业领域任务
SOP-Agent: Empower General Purpose AI Agent with Domain-Specific SOPs
- 用自然语言伪代码构建决策图，指导AI完成复杂任务
- 在多个领域任务中表现优于通用智能体，接近专用系统水平
- 专为客服场景设计首个基于SOP的评估基准
尽管通用AI智能体取得显著进展，但在实际应用中仍面临诸多挑战：大语言模型（LLM）的规划能力有限，难以处理需要长程规划的复杂任务；通用智能体也难以高效利用领域知识和人类经验。本文提出标准操作流程引导的智能体（SOP-agent），通过自然语言撰写的伪代码式标准操作流程（SOP）构建特定领域智能体。形式上，我们将SOP表示为决策图，通过遍历该图引导智能体完成指定任务。我们在多个领域进行了广泛实验，涵盖决策、搜索与推理、代码生成、数据清洗及基于真实情境的客户服务任务。SOP-agent展现出优异的泛化能力，在多数任务上性能优于通用智能体框架，且接近专用领域系统水平。此外，我们提出了首个基于SOP的地面化客户服务基准（Grounded Customer Service Benchmark），用于评估智能体在客户服务场景中的可解释决策能力。
原文摘要 · Abstract (English)
Despite significant advancements in general-purpose AI agents, several challenges still hinder their practical application in real-world scenarios. First, the limited planning capabilities of Large Language Models (LLM) restrict AI agents from effectively solving complex tasks that require long-horizon planning. Second, general-purpose AI agents struggle to efficiently utilize domain-specific knowledge and human expertise. In this paper, we introduce the Standard Operational Procedure-guided Agent (SOP-agent), a novel framework for constructing domain-specific agents through pseudocode-style Standard Operational Procedures (SOPs) written in natural language. Formally, we represent a SOP as a decision graph, which is traversed to guide the agent in completing tasks specified by the SOP. We conduct extensive experiments across tasks in multiple domains, including decision-making, search and reasoning, code generation, data cleaning, and grounded customer service. The SOP-agent demonstrates excellent versatility, achieving performance superior to general-purpose agent frameworks and comparable to domain-specific agent systems. Additionally, we introduce the Grounded Customer Service Benchmark, the first benchmark designed to evaluate the grounded decision-making capabilities of AI agents in customer service scenarios based on SOPs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。