轻量级视觉变压器，高效分割卫星影像时间序列。
VistaFormer: Scalable Vision Transformers for Satellite Image Time Series Segmentation
- 采用无位置编码的多尺度自注意力结构，简化模型并避免插值误差。
- 在PASTIS和MTLCC数据集上性能优于同类模型，计算量仅需8%-11%。
- 适合需要低资源高精度的遥感图像分割任务，尤其关注云噪声处理。
我们提出VistaFormer，一种基于轻量级Transformer的遥感图像语义分割模型。该模型采用多尺度Transformer编码器与轻量解码器，融合编码器中的全局与局部注意力。通过无位置编码的自注意力层，简化结构并消除时空编码插值需求，避免因训练测试分辨率差异导致的性能下降。研究了去除云等噪声输入信号的简单方法，并发现用邻域注意力（NA）替代多头自注意力（MHSA）可提升模型可扩展性。在PASTIS和MTLCC作物类型分割基准上的实验表明，VistaFormer在使用MHSA时仅需8%的浮点运算量，在使用NA时仅需11%，且参数更少。在PASTIS上相比现有最佳模型提升0.1%的平均交并比（mIoU），在MTLCC上分别提升3%（使用MHSA）和3.7%（使用NA）。
原文摘要 · Abstract (English)
We introduce VistaFormer, a lightweight Transformer-based model architecture for the semantic segmentation of remote-sensing images. This model uses a multi-scale Transformer-based encoder with a lightweight decoder that aggregates global and local attention captured in the encoder blocks. VistaFormer uses position-free self-attention layers which simplifies the model architecture and removes the need to interpolate temporal and spatial codes, which can reduce model performance when training and testing image resolutions differ. We investigate simple techniques for filtering noisy input signals like clouds and demonstrate that improved model scalability can be achieved by substituting Multi-Head Self-Attention (MHSA) with Neighbourhood Attention (NA). Experiments on the PASTIS and MTLCC crop-type segmentation benchmarks show that VistaFormer achieves better performance than comparable models and requires only 8% of the floating point operations using MHSA and 11% using NA while also using fewer trainable parameters. VistaFormer with MHSA improves on state-of-the-art mIoU scores by 0.1% on the PASTIS benchmark and 3% on the MTLCC benchmark while VistaFormer with NA improves on the MTLCC benchmark by 3.7%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。