arXiv:2608.17872cs.CV2026-08中稿 · ECCV

2200万参数模型通过知识蒸馏逼近百亿级病理模型性能,速度快25倍以上。

DistillPath: An Efficient 22M Distilled Pathology Encoder Approaching Large Foundation Model Performance

论文配图:DistillPath: An Efficient 22M Distilled Pathology Encoder Approaching Large Foundation Model Performance
图 1 · 摘自论文原文
  • 从4个大模型蒸馏特征,仅用6000张公开切片训练
  • 在3个基准上全优于基线,7项任务平均达0.795
  • 轻量高效,适合资源受限的病理图像分析场景

许多高性能病理切片编码器是参数量达数亿至上百亿的奠基模型。在普通硬件上用此类模型编码整张幻灯片的数千张切片成本高昂,因此保留下游性能的紧凑编码器极具价值。本文提出DistillPath-KS16,以2200万参数的kaiko ViT-S/16为基础,通过蒸馏已发布的病理编码器作为冻结教师模型进行优化。该方法仅读取教师模型的最终类别与切片标记,无需其DINO或iBOT预训练头,也不依赖十亿级切片数据集,适用于任何可访问骨干特征的发布模型。在6000张公共幻灯片上训练,将4个参数量从8600万到11亿不等的教师模型特征蒸馏至同一学生模型。所有变体均在三个基准(EVA、HEST、PLISM)上超越kaiko基线,最强教师表现具有任务依赖性。在七任务平均指标上,DistillPath-KS16-Virchow2达到0.795,距顶尖模型Virchow2的0.810仅差0.015,参数量仅为后者的约1/29;其综合性能高于H0-mini和GPFM,但优势集中在特定任务。因保持2200万参数与384维特征,运行速度比Virchow2快逾25倍。代码与模型权重已开源。

原文摘要 · Abstract (English)

Many high-performing pathology tile encoders are now foundation models with hundreds of millions to over a billion parameters. Encoding and storing the thousands of tiles in each whole-slide image with such models is costly on commodity hardware, so compact encoders that retain useful downstream performance are a valuable alternative. We present DistillPath-KS16, which starts from the existing 22M kaiko ViT-S/16 encoder and improves it by distilling from released pathology encoders used as frozen teachers. The recipe reads only the teachers' final class and patch tokens and trains on 6,000 public slides, needing neither their DINO nor iBOT pretraining heads nor a billion-tile corpus, so it applies to any released encoder that exposes backbone tokens. We distill four teachers spanning 86M to 1.1B parameters into the same student. Every variant improves the kaiko baseline on all three benchmarks we use, EVA, HEST, and PLISM, and the strongest teacher is task-dependent. On the seven-task EVA mean, DistillPath-KS16-Virchow2 reaches $0.795$, within $0.015$ points of Virchow2, the top-scoring model in our evaluation, at about $29\times$ fewer parameters; it also scores above H0-mini and GPFM on this aggregate metric, though that advantage is task-concentrated rather than uniform. Because it remains a 22M ViT-S/16 with 384-dimensional features, DistillPath-KS16 runs more than $25\times$ faster than Virchow2. Code is available at https://github.com/RamonKaspar/DistillPath, and released model weights are available at https://huggingface.co/collections/RamonK/distillpath.

病理图像知识蒸馏轻量化模型视觉编码器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。