arXiv:2607.23631cs.CV2026-07被引 1

用动态选词取代随机采样,让病理图像分析更准更快。

PathSelect: Sequential Token Selection for Whole Slide Pathology

论文配图:PathSelect: Sequential Token Selection for Whole Slide Pathology
图 1 · 摘自论文原文
  • 将切片令牌修剪重构为可学习的序列选择过程,自主决定保留哪些信息。
  • 仅用44.86个令牌(最大128)实现74.00%准确率,比原始数据少36.6倍令牌。
  • 适合需要高效处理超大病理图像的医学AI研究者与临床辅助系统开发者。

千兆像素全切片图像(WSI)因序列过长,严重制约视觉语言模型(VLM)的计算效率。现有方法多依赖空间采样或训练无关剪枝,易丢失分散分布的病理线索。本文将令牌剪枝重构成序列选择过程,使模型自主学习最优路由策略,而非依赖静态规则。提出PathSelect框架,作为SlideChat基模型的主动插件,冻结滑片编码器与大语言模型。为在训练中提供连续梯度,引入方差保持的噪声门,通过可微分的Soft Top-K操作调控每块信息流,并搭配对角注意力去噪器恢复扰动表示,避免语义泄露。推理时完全剥离PathSelect模块,仅用训练好的Scorer配合确定性Hard Top-K操作,实现自适应、数据依赖的终止路径,显著加速下游生成任务,令牌选择延迟极低。在SlideBench(TCGA)上,平均仅需44.86个令牌(上限K=128),达到74.00%整体准确率,相较未压缩基线减少约36.6倍空间令牌,持续优于采样类方法。

原文摘要 · Abstract (English)

Gigapixel Whole-Slide Images (WSIs) present a fundamental computational bottleneck for vision-language models (VLMs) due to extreme sequence lengths. Existing approaches predominantly rely on spatial sampling or training-free pruning, which risk diluting weak but informative signals, leading to the loss of critical diagnostic evidence due to the spatially diffuse nature of pathological cues. We reformulate WSI token pruning as a sequential selection process, enabling the model to autonomously learn an optimal routing strategy rather than relying on static heuristics. We herein propose a decoupled routing framework integrated as an active plugin into the fully pre-trained SlideChat base model, leaving both the slide encoder and large language model frozen. To provide continuous gradients for the non-differentiable pruning operation during training, we introduce PathSelect. PathSelect employs a variance-preserving noise gate to modulate each patch's information flow via a differentiable Soft Top-K operator, paired with a diagonal-attention Denoiser that recovers the perturbed representations without semantic leakage. At inference, the PathSelect module is entirely detached. Relying solely on the trained Scorer, a deterministic Hard Top-K operator executes adaptive, data-dependent trajectory termination, significantly accelerating downstream generative processing with exceptionally low sequential token selection latency. Driven by an empirical average of only 44.86 tokens under a maximum constraint of K = 128, our framework achieves 74.00% overall accuracy on SlideBench (TCGA), representing an approximate 36.6x spatial token reduction relative to the uncompressed baseline average while consistently outperforming sampling-based counterparts.

病理分析视觉语言令牌选择高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。