探究病理图像模型对旋转的不变性，发现旋转增强能显著提升性能。
Are the Latent Representations of Foundation Models for Pathology Invariant to Rotation?
- 用互k近邻和余弦距离量化旋转前后特征对齐度。
- 12个模型中，加入旋转增强的模型旋转不变性明显更强。
- 适合关注医学图像训练策略的研究者。
针对数字病理学的自监督基础模型将H&；E染色全切片图像的小块编码为用于下游任务的潜在表示。然而，这些表示对图像旋转的不变性尚未被研究。本研究通过互k近邻和余弦距离，量化了12个基础模型在非旋转与旋转图像块之间的特征对齐程度。结果显示，在自监督训练中引入旋转增强的模型表现出显著更高的旋转不变性。我们推测，由于Transformer架构缺乏旋转归纳偏置，必须在训练中加入旋转增强以实现学习到的不变性。代码已公开：https://github.com/MatousE/rot-invariance-analysis。
原文摘要 · Abstract (English)
Self-supervised foundation models for digital pathology encode small patches from H\&E whole slide images into latent representations used for downstream tasks. However, the invariance of these representations to patch rotation remains unexplored. This study investigates the rotational invariance of latent representations across twelve foundation models by quantifying the alignment between non-rotated and rotated patches using mutual $k$-nearest neighbours and cosine distance. Models that incorporated rotation augmentation during self-supervised training exhibited significantly greater invariance to rotations. We hypothesise that the absence of rotational inductive bias in the transformer architecture necessitates rotation augmentation during training to achieve learned invariance. Code: https://github.com/MatousE/rot-invariance-analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。