arXiv:2608.14309cs.CVq-bio.TO2026-08中稿 · MICCAI 2026 Worksh…

用语言空间传递信息,让模型读懂病理切片的局部结构关系。

Spatial Message Passing in Language Space for Pathology Image Interpretation

论文配图:Spatial Message Passing in Language Space for Pathology Image Interpretation
图 1 · 摘自论文原文
  • 将切片区域构造成文本图,通过语言消息传递实现空间推理。
  • 在多个数据集上提升肿瘤描述准确率3.3%至19.6%。
  • 无需微调模型权重,可自动优化决策规则,适合医学影像分析场景。

多模态大语言模型(MLLM)能从组织学图像生成病理描述,但千兆像素全切片图像(WSI)超出其视觉上下文限制。标准分块处理虽使图像可计算,却割裂了定义肿瘤-基质界面和形态的组织邻域关系。本文提出空间语言消息传递(SLMP)框架,完全在语言空间中进行空间推理,天然可读。SLMP将WSI区域表示为空间文本图:每块为节点,初始由MLLM生成描述,边编码空间邻接关系。每个节点通过共享聚合策略,整合相邻节点的语言消息,该策略作为可检查的提示,在网格上充当自适应局部核,作用于文本而非学习嵌入。该策略可通过观察组织表型变化,利用文本梯度自动优化,实现从局部细胞到整体组织形态的语义改进,无需微调MLLM权重。在代表性HER2和CAMELYON16区域测试中,SLMP在通用与病理专用骨干模型上均提升3.3%至19.6%的切片级肿瘤描述准确率。随机邻居消融实验表明,增益源于空间上下文而非额外文本。可视化优化策略揭示了可解释的、组织特异性的决策规则。此外,无需任何权重更新,SLMP显著提升通用MLLM性能,缩小与病理专用模型的差距,提供一种透明且灵活的空间推理机制。

原文摘要 · Abstract (English)

Multimodal Large Language Models (MLLMs) can generate pathological descriptions from histological images, but gigapixel Whole Slide Images (WSIs) exceed their visual context limits. The standard tiling workaround makes WSIs tractable yet severs the tissue neighborhoods that define tumor-stroma interfaces and morphology. We introduce Spatial Language Message Passing (SLMP), a framework that performs spatial reasoning entirely in language space, human-readable by construction. SLMP represents a WSI region as a spatial text graph: tiles are nodes initialized with MLLM descriptions, and edges encode spatial adjacency. For each tile, an LLM refines its description by integrating language messages from adjacent tiles under a shared aggregation policy that, on the tile grid, acts as an adaptive local kernel operating on text rather than learned embeddings. This policy is an inspectable prompt that can be refined from model-observed tissue phenotypes via textual gradients, enabling automatic semantic optimization from local cellular context to broader tissue morphology without fine-tuning MLLM weights. On representative HER2 and CAMELYON16 regions, SLMP improves tile-level tumor description accuracy in settings spanning general-purpose and pathology-specialized backbones, with gains of +3.3 to +19.6 percentage points. Random-neighbor ablations confirm that these gains stem from spatial context rather than additional text alone, and inspecting the optimized policies reveals interpretable, tissue-specific decision rules. Besides, without any weight updates or fine-tuning the backbone MLLM, SLMP substantially improves general-purpose MLLMs and narrows its gap to pathology-specialized counterparts, offering a transparent and flexible mechanism for incorporating spatial reasoning into MLLM-based pathology analysis.

病理分析语言模型空间推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。