PathFinder模拟专家病理医生,多智能体协作诊断皮肤癌切片。
PathFinder: A Multi-Modal Multi-Agent System for Medical Diagnostic Decision-Making Applied to Histopathology
- 四智能体协同:分诊、导航、描述、诊断,逐步聚焦关键区域。
- 皮肤癌诊断准确率比顶尖方法高8%,首次超过人类病理医生平均表现9%。
- 输出自然语言解释,结果可解释性强,适合临床辅助决策场景。
通过全幻灯片图像(WSI)进行疾病诊断是现代病理学的基础,但受制于瓦级像素规模与复杂性。经验丰富的病理医生通过迭代式导航、标记关键区域并整合信息来完成诊断。传统人工智能方法如多实例学习和基于Transformer的模型难以模拟这一全过程,限制了其在实际中的应用。本文提出PathFinder,一种多模态多智能体框架,模仿专家病理医生的诊断流程。该系统包含四个智能体:分诊智能体判断病变风险;若为高风险,则导航与描述智能体协同聚焦显著区域,生成重要性图谱并提供采样区域的描述性见解;最终由诊断智能体综合分析得出诊断结论。实验表明,PathFinder在皮肤黑色素瘤诊断上比现有最佳方法提升8%,且首次在该任务中超越人类病理医生平均水平9%。其输出具备自然语言解释能力,路径清晰可追溯。病理医生定性评估显示描述质量媲美GPT-4o。相关数据、代码与模型已公开于https://pathfinder-dx.github.io/
原文摘要 · Abstract (English)
Diagnosing diseases through histopathology whole slide images (WSIs) is fundamental in modern pathology but is challenged by the gigapixel scale and complexity of WSIs. Trained histopathologists overcome this challenge by navigating the WSI, looking for relevant patches, taking notes, and compiling them to produce a final holistic diagnostic. Traditional AI approaches, such as multiple instance learning and transformer-based models, fail short of such a holistic, iterative, multi-scale diagnostic procedure, limiting their adoption in the real-world. We introduce PathFinder, a multi-modal, multi-agent framework that emulates the decision-making process of expert pathologists. PathFinder integrates four AI agents, the Triage Agent, Navigation Agent, Description Agent, and Diagnosis Agent, that collaboratively navigate WSIs, gather evidence, and provide comprehensive diagnoses with natural language explanations. The Triage Agent classifies the WSI as benign or risky; if risky, the Navigation and Description Agents iteratively focus on significant regions, generating importance maps and descriptive insights of sampled patches. Finally, the Diagnosis Agent synthesizes the findings to determine the patient's diagnostic classification. Our Experiments show that PathFinder outperforms state-of-the-art methods in skin melanoma diagnosis by 8% while offering inherent explainability through natural language descriptions of diagnostically relevant patches. Qualitative analysis by pathologists shows that the Description Agent's outputs are of high quality and comparable to GPT-4o. PathFinder is also the first AI-based system to surpass the average performance of pathologists in this challenging melanoma classification task by 9%, setting a new record for efficient, accurate, and interpretable AI-assisted diagnostics in pathology. Data, code and models available at https://pathfinder-dx.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。