arXiv:2410.17657cs.CL2024-10被引 28

让医疗智能体学会反思,用经验指导工具使用。

ReflecTool: Towards Reflection-Aware Tool-Augmented Clinical Agents

  • 分两阶段优化:存成功经验,推理时调用记忆选工具。
  • 在18项临床任务中比纯LLM高10+分,比现有方法高3分。
  • 适合需要多工具协同的复杂医疗决策场景。

大语言模型在医疗领域展现出潜力,如辅助病历生成和患者沟通。然而,现有模型仅限于文本交互,难以处理临床环境中的多元信息。尽管已有临床智能体能处理多种信号,但它们仅针对单一临床场景,缺乏泛化能力。为此,我们提出ClinicalAgent Bench(CAB),一个包含18项任务、覆盖五大真实临床维度的综合性医疗智能体评估基准。基于此,我们引入ReflecTool框架,在两个阶段中高效利用领域专用工具:第一阶段通过保存小规模预定义训练集中的成功求解过程与工具经验,逐步扩展长期记忆;第二阶段推理时,可从已构建的长期记忆中检索支持性成功范例,指导工具选择策略,并通过迭代精炼与候选选择两种验证方法改进工具使用。在ClinicalAgent Bench上的大量实验表明,ReflecTool在复杂临床任务中表现优于纯大语言模型超过10分,优于现有代理方法3分,凸显其适应性与有效性。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have shown promising potential in the medical domain, assisting with tasks like clinical note generation and patient communication. However, current LLMs are limited to text-based communication, hindering their ability to interact with diverse forms of information in clinical environments. Despite clinical agents succeeding in diverse signal interaction, they are oriented to a single clinical scenario and hence fail for broader applications. To evaluate clinical agents holistically, we propose ClinicalAgent Bench~(CAB), a comprehensive medical agent benchmark consisting of 18 tasks across five key realistic clinical dimensions. Building on this, we introduce ReflecTool, a novel framework that excels at utilizing domain-specific tools within two stages. The first optimization stage progressively enlarges a long-term memory by saving successful solving processes and tool-wise experience of agents in a tiny pre-defined training set. In the following inference stage, ReflecTool can search for supportive successful demonstrations from already built long-term memory to guide the tool selection strategy, and a verifier improves the tool usage according to the tool-wise experience with two verification methods--iterative refinement and candidate selection. Extensive experiments on ClinicalAgent Benchmark demonstrate that ReflecTool surpasses the pure LLMs with more than 10 points and the well-established agent-based methods with 3 points, highlighting its adaptability and effectiveness in solving complex clinical tasks.

医疗AI工具增强反思机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。