用可训练张量分解，让3通道模型适配高光谱图像
Transfer learning RGB models to hyperspectral images with trainable tensor decompositions
- 将预训练卷积核分解为空间与光谱成分，仅替换光谱部分
- 在多个高光谱数据集上准确率优于现有迁移学习方法
- 适合做高光谱图像分类的工程师和研究人员
迁移学习使大型视觉网络能通过调整通用滤波器适应新任务。但这些网络假设输入为3通道图像，无法直接处理多光谱或高光谱图像。现有方法在图像或模型信息上有所损失。本文提出一种新方法，通过部分可训练张量分解，保留图像与模型的空间信息。我们对预训练卷积滤波器进行分解，分离出空间与光谱成分，并将光谱成分替换为更高通道维度的可训练组件，生成可适应新数据集的高光谱滤波器，同时保持原始滤波器的空间模式。在多个高光谱数据集上的实验表明，该方法比其他高光谱迁移学习方法更准确且更鲁棒。
原文摘要 · Abstract (English)
Transfer learning makes it possible to use large vision networks on a variety of domains, by specializing their models' general filters to new tasks. However, these networks assume the input images to have 3 input channels, making them incompatible with multi- or hyperspectral images. Current approaches that mitigate this incompatibility sacrifice information in either the image, or the model. This work proposes a novel approach that preserves the image and spatial information present in the model by using partially trainable tensor decompositions. We create such decompositions of pretrained convolutional filters, separating the filters into spatial and spectral components. The spectral components are then replaced with trainable components of higher channel dimensionality. This creates hyperspectral filters that can specialize to new datasets, while retaining the spatial patterns of the original filter. Experiments on a variety of hyperspectral datasets show that our approach is more accurate and robust than other hyperspectral transfer learning methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。