对比两种医学推理方法，发现结合使用效果最佳。
MedicalAgentsBench for Complex Medical Reasoning: Comparing Internalized Reasoning Models versus Externalized Agent-based Frameworks
- 用多模型协作框架分解复杂医学问题
- 组合使用内部推理与外部代理模型，准确率达35.1%
- 适合医疗AI研究者和资源有限的团队参考
复杂医学推理需要在多个推理步骤中整合异构临床证据。大语言模型（LLMs）目前通过两种路径实现：内部化推理和外部化代理架构（由多个LLM协同分解问题）。为判断这两种路径是互斥还是互补，我们提出MedicalAgentsBench，一个经过难度感知筛选和污染检测的862个复杂临床问题基准集，来自八个医学数据集的合并。评估了三个内部化推理模型（DeepSeek-R1、o1-mini、o3-mini）、七个基础模型和九种外部化代理方法，发现内部化与外部化方法各自独立提升性能，且其优势可叠加：将代理工作流嵌入内部化推理模型（如o3-mini + MDAgents）达到最高准确率35.1%。帕累托分析显示该组合在成本-性能前沿占据主导；同时，对低成本模型的轻量优化为资源受限场景提供了可行入口。基准代码已公开于https://github.com/gersteinlab/MedicalAgentsBench。
原文摘要 · Abstract (English)
Complex medical reasoning requires integrating heterogeneous clinical evidence across multiple inference steps. Large language models (LLMs) now approach this through two routes: internalized reasoning and externalized agent scaffolding (frameworks that decompose problems collaboratively amongst multiple LLMs). To determine whether these routes are exclusive or complementary, we introduce MedicalAgentsBench, a filtered benchmark of 862 complex clinical questions drawn from the union of eight medical datasets via difficulty-aware curation and contamination screening. Evaluating three internalized reasoning models (DeepSeek-R1, o1-mini, and o3-mini), seven base models, and nine externalized agent-based methods, we find that internalized and externalized approaches each independently improve performance, and that their benefits compound: the highest accuracy is achieved by layering agent workflows onto an internalized reasoning model (i.e., o3-mini + MDAgents with 35.1%). Pareto analysis shows this combination dominates the cost-performance frontier; moreover, lightweight optimization on inexpensive models offers an entry point for resource-constrained settings. Our benchmark is at https://github.com/gersteinlab/MedicalAgentsBench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。