用卷积网络提升图像特征分辨率，实现材料显微图高效精准分割
Maybe you don't need a U-Net: convolutional feature upsampling for materials micrograph segmentation
- 用卷积网络对低分辨率特征进行上采样，结合输入图像恢复细节
- 在植物细胞、电池正极等图像上实现高质量分割，可识别细小裂纹
- 交互式分割速度快、标注少，无需额外训练，适合材料图像分析
特征基础模型（通常为视觉变换器）能提供丰富的图像语义描述，适用于下游任务如（交互式）分割和目标检测。但由于这些描述基于图像块，分辨率较低，难以表征材料与生物图像中常见的细微结构，且在大尺寸图像上计算效率较低。本文训练了一个卷积神经网络，利用输入图像作为参考，对低分辨率（即大块尺寸）的基础模型特征进行上采样。该上采样网络在不需进一步训练的情况下，可高效提取特征并分割多种显微图像，包括植物细胞、锂离子电池正极和有机晶体。上采样后的特征丰富度使难以分割的相（如细小裂纹）得以分离。实验表明，使用这些深度特征进行交互式分割，可在更短时间内、仅用极少标注生成高质量分割结果，远优于传统卷积网络的训练或微调方式。
原文摘要 · Abstract (English)
Feature foundation models - usually vision transformers - offer rich semantic descriptors of images, useful for downstream tasks such as (interactive) segmentation and object detection. For computational efficiency these descriptors are often patch-based, and so struggle to represent the fine features often present in micrographs; they also struggle with the large image sizes present in materials and biological image analysis. In this work, we train a convolutional neural network to upsample low-resolution (i.e, large patch size) foundation model features with reference to the input image. We apply this upsampler network (without any further training) to efficiently featurise and then segment a variety of microscopy images, including plant cells, a lithium-ion battery cathode and organic crystals. The richness of these upsampled features admits separation of hard to segment phases, like hairline cracks. We demonstrate that interactive segmentation with these deep features produces high-quality segmentations far faster and with far fewer labels than training or finetuning a more traditional convolutional network.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。