arXiv:2607.19624cs.CV2026-07中稿 · publication at the…

让AI报告生成更像病理医生,通过模仿医生看片注意力提升准确性。

Pathologist Attention-Aligned Report Generation for Prostate Histopathology

论文配图:Pathologist Attention-Aligned Report Generation for Prostate Histopathology
图 1 · 摘自论文原文
  • 用医生看片时的视线轨迹训练AI,让模型关注区域与医生一致。
  • 报告生成准确率提升19.3%,语言评价指标平均提高10.9%。
  • 生成报告时自动高亮关键区域,适合临床辅助诊断场景。

病理医生在癌症诊断中对全切片图像(WSIs)的视觉注意力高度选择性,深刻影响信息提取。这种注意力是识别诊断关键区域的重要语义线索,可用于报告生成。本文将人类注意力引入病理报告生成模型的训练过程。我们构建了一个包含121例前列腺WSI的多模态注意力数据集,标注了医生在五个临床相关组件(如格里森分级模式)上的多尺度视窗轨迹、口头描述及光标移动。基于该数据集,采用注意力对齐损失微调两个报告生成模型,使其图像块注意力分布匹配病理医生的注意力分布。在前列腺癌报告生成和视觉问答任务上评估了两种具有不同内部注意力机制的模型。实验显示,在五项临床相关报告成分上,平均提升10.9%的NLP指标得分,准确率提升19.3%。此外,推理时生成的注意力热图与病理医生注意力更接近,仅需极低计算开销,为报告提供更强视觉支持。

原文摘要 · Abstract (English)

The allocation of visual attention by pathologists during cancer diagnosis is a highly selective process that critically shapes the information extracted from whole-slide images (WSIs). Human attention helps medical imaging tasks such as classification and segmentation, and becomes a strong semantic cue for identifying diagnostically informative regions for report generation. In this paper, we introduce human attention into the training of pathologist report generation models. To this end, we collected a multimodal human-attention dataset of 121 prostate WSIs annotated with pathologists' multi-scale viewport trajectories synchronized with the pathologists' verbal descriptions and cursor movements for five clinically relevant components (e.g., Gleason patterns). Using this dataset, we finetune two report generation models with an attention-alignment loss that regularizes the model attention over image patches to match the distribution of pathologist attention. We evaluate our approach on prostate cancer report generation and visual question answering using two models with different internal attention mechanisms (i.e., how image tokens are integrated into the language decoder). Experiments show average gains of 10.9% on NLP-based metrics and 19.3% in accuracy across five clinically relevant report components. Further, model attention maps extracted at inference time, with minimal computational overhead, align more closely with pathologist attention, providing stronger visual support for the generated reports by highlighting the regions that most influence the output.

病理报告注意力对齐前列腺癌

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。