MuViT通过统一坐标系融合多尺度显微图像，提升分析精度。
MuViT: Multi-Resolution Vision Transformers for Learning Across Scales in Microscopy
- 将不同分辨率的图像块映射到统一坐标系，实现跨尺度融合。
- 在肾组织病理和小鼠脑部显微图像上，性能优于ViT和CNN基线。
- 支持多尺度自监督预训练，适合大规模显微图像分析任务。
现代显微技术常生成包含多空间尺度的吉字节级图像，从细胞细微结构到组织整体布局均需分析。多数视觉模型仅处理单一分辨率或从单视角提取多尺度特征，难以充分利用显微数据的固有多分辨率特性。我们提出MuViT，一种专为融合同一图像的真实多分辨率观测而设计的Transformer架构。该模型将所有图像块嵌入共享的世界坐标系，并扩展旋转位置编码以支持该坐标，使注意力机制可在单一编码器中整合广域上下文与高分辨率细节。在合成基准、肾组织病理学及高分辨率小鼠脑显微图像上，MuViT持续优于强基准的ViT和CNN模型。多尺度MAE自监督预训练进一步生成尺度一致的表征，显著提升下游任务表现。结果表明，显式世界坐标建模为大规模显微图像分析中利用多分辨率信息提供了简单而强大的机制。
原文摘要 · Abstract (English)
Modern microscopy routinely produces gigapixel images that contain structures across multiple spatial scales, from fine cellular morphology to broader tissue organization. Many analysis tasks require combining these scales, yet most vision models operate at a single resolution or derive multi-scale features from one view, limiting their ability to exploit the inherently multi-resolution nature of microscopy data. We introduce MuViT, a transformer architecture built to fuse true multi-resolution observations from the same underlying image. MuViT embeds all patches into a shared world-coordinate system and extends rotary positional embeddings to these coordinates, enabling attention to integrate wide-field context with high-resolution detail within a single encoder. Across synthetic benchmarks, kidney histopathology, and high-resolution mouse-brain microscopy, MuViT delivers consistent improvements over strong ViT and CNN baselines. Multi-resolution MAE pretraining further produces scale-consistent representations that enhance downstream tasks. These results demonstrate that explicit world-coordinate modelling provides a simple yet powerful mechanism for leveraging multi-resolution information in large-scale microscopy analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。