提升病理图像分析效率与精度，让大模型更懂病灶细节。
Efficient and Comprehensive Feature Extraction in Large Vision-Language Model for Pathology Analysis
- 通过任务引导增强特征提取，聚焦多尺度病灶区域。
- 在不降速前提下融合粗细粒度特征，准确率显著提升。
- 适用于多种病理诊断场景，临床辅助价值高。
病理诊断对疾病特征判断、治疗指导和预后评估至关重要，依赖于对高分辨率全切片图像（WSI）的详细多尺度分析。然而，现有大规模视觉语言模型（LVLM）受输入分辨率限制，影响其在病理图像分析中的效率与准确性。为此，我们提出两种创新策略：混合任务引导的特征增强，使特征提取聚焦跨尺度病变细节；提示引导的细节特征补全，基于特定提示整合WSI的粗细粒度特征，且不牺牲推理速度。利用包含49万样本的多样化病理任务数据集，我们训练了专用于病理的LVLM——OmniPath。大量实验表明，该模型在诊断准确率和效率上显著优于现有方法，为多种病理应用提供了交互式、临床对齐的辅助诊断方案。
原文摘要 · Abstract (English)
Pathological diagnosis is vital for determining disease characteristics, guiding treatment, and assessing prognosis, relying heavily on detailed, multi-scale analysis of high-resolution whole slide images (WSI). However, existing large vision-language models (LVLMs) are limited by input resolution constraints, hindering their efficiency and accuracy in pathology image analysis. To overcome these issues, we propose two innovative strategies: the mixed task-guided feature enhancement, which directs feature extraction toward lesion-related details across scales, and the prompt-guided detail feature completion, which integrates coarse- and fine-grained features from WSI based on specific prompts without compromising inference speed. Leveraging a comprehensive dataset of 490K samples from diverse pathology tasks, we trained the pathology-specialized LVLM, OmniPath. Extensive experiments demonstrate that this model significantly outperforms existing methods in diagnostic accuracy and efficiency, providing an interactive, clinically aligned approach for auxiliary diagnosis in a wide range of pathology applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。