让AI像医生一样思考，自动定位癌症切片中的关键区域。
PathReasoning: A multimodal reasoning agent for query-based ROI navigation on whole-slide images
- 通过多轮自我反思与推理，逐步聚焦诊断相关区域
- 在肿瘤分型和随访分析任务上分别提升6.7%和3.1% AUROC
- 适合需要精准定位与可解释性报告的数字病理场景
从全幻灯片图像（WSIs）中解读肿瘤微环境对癌症诊断、预后和治疗反应至关重要。然而，这些高达10亿像素以上的超大图像，使得定位特定区域进行临床检查既耗时又困难。受病理医生通过采样、推理与自我反思导航图像的启发，我们提出「PathReasoning」——一个支持多模态推理的智能代理，通过多轮迭代推理与优化，在WSIs中逐步导航至关键区域。初始随机采样候选区域后，PathReasoning通过自省评估当前选择，结合视觉观察与临床问题进行推理，并生成新探索区域。多轮迭代构建出指向诊断相关区域的推理链。该方法将整张幻灯片转化为一系列问题引导的视图，在固定步数内高效定位信息丰富的ROI，无需密集像素级标注。在肿瘤分型与纵向分析任务中，相比强基线方法，分别提升6.7%与3.1%的AUROC；高质量ROI进一步支持乳腺癌报告生成，准确率比标准GPT-4o高出10%。PathReasoning优先关注问题相关的区域，构建可解释的推理链，支持高效阅片、一致诊断、全面报告与证据追溯，推动数字病理智能化发展。
原文摘要 · Abstract (English)
Deciphering tumor microenvironment from Whole Slide Images (WSIs) is intriguing as it is key to cancer diagnosis, prognosis and treatment response. While these gigapixel images on one hand offer a comprehensive portrait of cancer, on the other hand, the extremely large size, as much as more than 10 billion pixels, make it challenging and time-consuming to navigate to corresponding regions to support diverse clinical inspection. Inspired by pathologists who conducted navigation on WSIs with a combination of sampling, reasoning and self-reflection, we proposed "PathReasoning", a multi-modal reasoning agent that iteratively navigates across WSIs through multiple rounds of reasoning and refinements. Specifically, starting with randomly sampled candidate regions, PathReasoning reviews current selections with self-reflection, reasoning over the correspondence between visual observations and clinical questions, and concludes by proposing new regions to explore. Across rounds, PathReasoning builds a reasoning chain that gradually directs attention to diagnostically relevant areas. PathReasoning turns each whole slide into a sequence of question-guided views, allowing the model to efficiently find informative ROIs within a fixed number of steps, without the need for dense pixel-level annotations. PathReasoning can substantially outperform strong ROI-selection approaches by 6.7% and 3.1% of AUROC on subtyping and longitudinal analysis tasks. The high-quality ROIs further support accurate report generation on breast cancer, significantly outperforming the standard GPT-4o by 10% in accuracy. PathReasoning prioritizes question-specific regions and constructs interpretable reasoning chains, supporting efficient slide review, consistent diagnostic interpretations, comprehensive reporting, and evidence traceability in digital pathology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。