arXiv:2602.21517cs.CV2026-02

让AI医生学会判断不同工具的可信度,自动选最靠谱的诊断结果。

Which Tool Response Should I Trust? Tool-Expertise-Aware Chest X-ray Agent with Multimodal Agentic Learning

  • 通过实验试错学习各工具在不同问题上的真实可靠性
  • 在胸部X光分析中超越现有方法,准确率显著提升
  • 适合需要多工具协同决策的医疗AI研究者

具备工具调用能力的AI代理在整合多种工具领域专长方面展现出潜力。然而,在医疗领域中,工具通常是内在错误率较高的AI模型,可能产生相互矛盾的结果。现有医学代理研究缺乏对工具实际可靠性的充分理解,因而无法有效解决工具冲突。为此,本文提出一个框架,使代理能够与工具交互,并通过代理式学习,在不同多模态查询类型下实证性地学习工具的实际可信度。作为具体实例,我们聚焦于胸部X光分析,提出了一个工具专长感知的胸部X光代理(TEA-CXA)。当工具输出不一致时,代理会实验性地接受或拒绝多模态工具结果,获取奖励,并学习针对每种查询类型应信任哪个工具。重要的是,TEA-CXA将现有的多轮工具调用强化学习代码库扩展至支持多模态上下文,同时增强了医疗场景下的功能:支持单轮多次工具调用、并行工具推理及单个用户查询中的多图像处理。该代码框架适用于一般医学研究中的多轮多模态工具调用强化学习。实验表明,TEA-CXA在性能上优于当前最先进的方法和一系列基线模型。代码将公开发布。

原文摘要 · Abstract (English)

AI agents with tool-use capabilities show promise for integrating the domain expertise of various tools. In the medical field, however, tools are usually AI models that are inherently error-prone and can produce contradictory responses. Existing research on medical agents lacks sufficient understanding of the tools' realistic reliability and thus cannot effectively resolve tool conflicts. To address this gap, this paper introduces a framework that enables an agent to interact with tools and empirically learn their practical trustworthiness across different types of multimodal queries via agentic learning. As a concrete instantiation, we focus on chest X-ray analysis and present a tool-expertise-aware chest X-ray agent (TEA-CXA). When tool outputs disagree, the agent experimentally accepts or rejects multimodal tool results, receives rewards, and learns which tool to trust for each query type. Importantly, TEA-CXA extends existing codebases for reinforcement learning with multi-turn tool-calling that focus on textual inputs, to support multimodal contexts effectively. In addition, we enhance the codebase for medical use scenarios by supporting multiple tool calls in one turn, parallel tool inference, and multi-image accommodation within a single user query. Our code framework is applicable to general medical research on multi-turn tool-calling reinforcement learning in multimodal settings. Experiments show that TEA-CXA outperforms the state-of-the-art methods and a comprehensive set of baselines. Code will be released.

医疗AI工具调用多模态强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。