融合高光谱与多时相数据,提升作物细粒度分类精度。
Fine-grained Hierarchical Crop Type Classification from Integrated Hyperspectral EnMAP Data and Multispectral Sentinel-2 Time Series: A Large-scale Dataset and Dual-stream Transformer Method
- 双流Transformer分别处理高光谱与时间序列数据
- 相比仅用哨兵2号数据,平均F1提升4.2%(最高达6.3%)
- 适用于需要精细作物分类的农业遥感研究
细粒度作物类型分类是大规模作物制图的基础,对保障粮食安全至关重要。它需同时捕捉物候动态(来自哨兵2号等多时相卫星数据)和细微光谱变化(依赖纳米级分辨率的高光谱影像)。由于高光谱数据获取难、作物标注成本高,当前跨模态研究仍较少。为此,我们整合30米分辨率的EnMAP高光谱数据与哨兵2号时序数据,构建了层级化高光谱作物数据集H2Crop。该数据集包含超百万个标注田块,采用四级作物分类体系,为细粒度农业作物分类与高光谱图像处理提供重要基准。我们提出双流Transformer架构,协同处理两种模态:光谱-空间Transformer从EnMAP数据中提取精细特征,时序Swin Transformer从哨兵2号时序中捕捉生长模式。设计的层级分类头与层级融合机制可同时输出各分类层级的结果。实验表明,加入EnMAP数据使平均F1得分提升4.2%(峰值达6.3%)。大量对比验证了本方法在各类时间窗口与作物变化场景下均优于现有深度学习模型,且持续受益于高光谱信息。代码与数据集见https://github.com/flyakon/H2Crop。
原文摘要 · Abstract (English)
Fine-grained crop type classification serves as the fundamental basis for large-scale crop mapping and plays a vital role in ensuring food security. It requires simultaneous capture of both phenological dynamics (obtained from multi-temporal satellite data like Sentinel-2) and subtle spectral variations (demanding nanometer-scale spectral resolution from hyperspectral imagery). Research combining these two modalities remains scarce currently due to challenges in hyperspectral data acquisition and crop types annotation costs. To address these issues, we construct a hierarchical hyperspectral crop dataset (H2Crop) by integrating 30m-resolution EnMAP hyperspectral data with Sentinel-2 time series. With over one million annotated field parcels organized in a four-tier crop taxonomy, H2Crop establishes a vital benchmark for fine-grained agricultural crop classification and hyperspectral image processing. We propose a dual-stream Transformer architecture that synergistically processes these modalities. It coordinates two specialized pathways: a spectral-spatial Transformer extracts fine-grained signatures from hyperspectral EnMAP data, while a temporal Swin Transformer extracts crop growth patterns from Sentinel-2 time series. The designed hierarchical classification head with hierarchical fusion then simultaneously delivers multi-level crop type classification across all taxonomic tiers. Experiments demonstrate that adding hyperspectral EnMAP data to Sentinel-2 time series yields a 4.2% average F1-scores improvement (peaking at 6.3%). Extensive comparisons also confirm our method's higher accuracy over existing deep learning approaches for crop type classification and the consistent benefits of hyperspectral data across varying temporal windows and crop change scenarios. Codes and dataset are available at https://github.com/flyakon/H2Crop.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。