用多个专家智能体协作分析病理切片,兼顾准确与多任务能力
WSI-Agents: A Collaborative Multi-Agent System for Multi-Modal Whole Slide Image Analysis
- 设计专家智能体分工协作,按任务自动分配模型
- 通过内部一致性检查和病理知识库验证,提升结果可信度
- 适合需要多任务精准分析的数字病理研究者
全切片图像(WSI)在数字病理中至关重要,支持跨多种病理任务的千兆像素组织分析。尽管多模态大语言模型(MLLM)可通过自然语言实现多任务WSI分析,但其性能常低于专用模型。协作式多智能体系统在医疗领域展现潜力,但在病理学中的应用仍不充分。为此,我们提出WSI-Agents,一种用于多模态WSI分析的新型协作多智能体系统。该系统整合专用功能智能体与强任务分配、验证机制,通过三个组件提升任务特异性准确率与多任务通用性:(1) 任务分配模块利用补丁级与全切片级MLLM模型库,将任务分配给专家智能体;(2) 验证机制通过内部一致性检查与病理知识库及领域专用模型的外部验证确保准确性;(3) 汇总模块生成最终摘要并附带可视化解释图。在多模态WSI基准测试上,WSI-Agents在多样任务中均优于现有WSI MLLMs与医学智能体框架。
原文摘要 · Abstract (English)
Whole slide images (WSIs) are vital in digital pathology, enabling gigapixel tissue analysis across various pathological tasks. While recent advancements in multi-modal large language models (MLLMs) allow multi-task WSI analysis through natural language, they often underperform compared to task-specific models. Collaborative multi-agent systems have emerged as a promising solution to balance versatility and accuracy in healthcare, yet their potential remains underexplored in pathology-specific domains. To address these issues, we propose WSI-Agents, a novel collaborative multi-agent system for multi-modal WSI analysis. WSI-Agents integrates specialized functional agents with robust task allocation and verification mechanisms to enhance both task-specific accuracy and multi-task versatility through three components: (1) a task allocation module assigning tasks to expert agents using a model zoo of patch and WSI level MLLMs, (2) a verification mechanism ensuring accuracy through internal consistency checks and external validation using pathology knowledge bases and domain-specific models, and (3) a summary module synthesizing the final summary with visual interpretation maps. Extensive experiments on multi-modal WSI benchmarks show WSI-Agents's superiority to current WSI MLLMs and medical agent frameworks across diverse tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。