通过精巧工具接口设计，让大模型在复杂客服任务中表现大幅提升。
The Art of Tool Interface Design
- 用状态机结构化业务逻辑，让大模型像调用工具一样使用。
- 不微调模型，在真实零售场景中成功率提升至82.6%（GPT-4o）。
- 适合需要高可靠推理的智能客服、自动化流程系统开发者。
我们提出一个名为Thinker的代理框架，在涉及复杂业务逻辑和人类交互的长周期客户服务业务场景中，实现了当前最优的推理性能。在τ-bench零售数据集上，Thinker使用GPT-4o（2024-06-01版本）取得82.6%的成功率（基线68.3%），使用Llama-3.1 405B取得81.9%的成功率（基线49.6%），且无需任何微调。Thinker通过引入恰当结构，有效缩小了基础模型间的推理能力差距。其核心特性包括：(1) 状态机增强生成（SMAG），将业务逻辑表示为状态机，由大模型作为工具调用；(2) 将主推理循环中的任务委派给大模型驱动的工具；(3) 自适应上下文管理。该仅靠提示词的设计方案在保持标准ReAct风格代理架构的同时，实现了显著性能提升，关键在于工具接口设计的创新，如SMAG与大模型工具的结合。
原文摘要 · Abstract (English)
We present an agentic framework, Thinker, which achieves state of art performance in challenging reasoning tasks for realistic customer service scenarios that involve complex business logic and human interactions via long horizons. On the $τ$-bench retail dataset, Thinker achieves 82.6\% success rate with GPT-4o (version 2024-06-01) (baseline: 68.3\%), and 81.9\% success rate with Llama-3.1 405B (baseline: 49.6\%), without any fine-tuning. Thinker effectively closes the gap in reasoning capabilities between the base models by introducing proper structure. The key features of the Thinker framework are: (1) State-Machine Augmented Generation (SMAG), which represents business logic as state machines and the LLM uses state machines as tools. (2) Delegation of tasks from the main reasoning loop to LLM-powered tools. (3) Adaptive context management. Our prompting-only solution achieves signficant gains, while still maintaining a standard agentic architecture with a ReAct style reasoning loop. The key is to innovate on the tool interface design, as exemplified by SMAG and the LLM-powered tools.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。