无需真实标签，通过三维度一致性评估病理视觉语言模型可靠性。
PathGLS: Evaluating Pathology Vision-Language Models without Ground Truth through Multi-Dimensional Consistency
- 构建多维一致性评估框架，从对齐、逻辑和稳定性三方面检测幻觉。
- 在Quilt-1M上发现幻觉报告敏感度下降40.2%，远超BERTScore的2.1%。
- 可适配切片与全幻灯片分析，适合临床部署前模型可信度验证。
视觉语言模型（VLMs）在计算病理学中具有重要潜力，可实现可解释图像分析、自动化报告和规模化决策支持。然而，由于缺乏可靠的自动化评估指标，难以识别细微错误（如幻觉），限制了其临床应用。为此，我们提出PathGLS，一种新颖的无参考评估框架，从三个维度评估病理VLMs：定位性（细粒度图文对齐）、逻辑性（基于自然语言推理的蕴含图一致性）和稳定性（对抗性视听扰动下的输出方差）。该框架支持切片级和全幻灯片级分析，生成综合可信度评分。在Quilt-1M、TCGA、REG2025、PathMMU及TCGA-Sarcoma数据集上的实验表明，PathGLS表现优异。尤其在Quilt-1M上，其对幻觉报告的敏感度下降达40.2%，远高于BERTScore的2.1%。与专家定义的临床错误层级对比，PathGLS获得Spearman秩相关系数ρ=0.71（p < 0.0001），显著优于基于大语言模型的方法（Gemini 3.0 Pro: ρ=0.39，p < 0.0001）。结果证明PathGLS是一种鲁棒的无参考度量方法，能直接量化幻觉率与领域偏移鲁棒性，适用于私有临床数据集的模型基准测试与安全部署指导。代码见：https://github.com/My13ad/PathGLS
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) offer significant potential in computational pathology by enabling interpretable image analysis, automated reporting, and scalable decision support. However, their widespread clinical adoption remains limited due to the absence of reliable, automated evaluation metrics capable of identifying subtle failures such as hallucinations. To address this gap, we propose PathGLS, a novel reference-free evaluation framework that assesses pathology VLMs across three dimensions: Grounding (fine-grained visual-text alignment), Logic (entailment graph consistency using Natural Language Inference), and Stability (output variance under adversarial visual-semantic perturbations). PathGLS supports both patch-level and whole-slide image (WSI)-level analysis, yielding a comprehensive trust score. Experiments on Quilt-1M, TCGA, REG2025, PathMMU and TCGA-Sarcoma datasets demonstrate the superiority of PathGLS. Specifically, on the Quilt-1M dataset, PathGLS reveals a steep sensitivity drop of 40.2% for hallucinated reports compared to only 2.1% for BERTScore. Moreover, validation against expert-defined clinical error hierarchies reveals that PathGLS achieves a strong Spearman's rank correlation of $ρ=0.71$ ($p < 0.0001$), significantly outperforming Large Language Model (LLM)-based approaches (Gemini 3.0 Pro: $ρ=0.39$, $p < 0.0001$). These results establish PathGLS as a robust reference-free metric. By directly quantifying hallucination rates and domain shift robustness, it serves as a reliable criterion for benchmarking VLMs on private clinical datasets and informing safe deployment. Code can be found at: https://github.com/My13ad/PathGLS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。