针对遥感图像分割,按类别动态融合多个模型预测结果。
ClassWise-CRF: Category-Specific Fusion for Enhanced Semantic Segmentation of Remote Sensing Imagery
- 从多个模型中选优,按类别加权融合预测结果。
- 在LoveDA和Vaihingen数据集上mIoU提升0.68%~1.00%。
- 适合需要高精度分类的遥感图像分析任务。
我们提出一种名为ClassWise-CRF的结果级类别特定融合架构。该架构采用两阶段流程:首先通过贪心算法从候选网络池中筛选出在特定类别表现优异的专家网络;其次,基于各网络在每类上的分割性能,自适应加权融合其预测结果。受条件随机场(CRF)启发,ClassWise-CRF将多网络的分割预测视为置信度场,利用验证集上的分割指标(如交并比,IoU)作为先验,并采用指数加权策略融合各网络的类别置信度。该融合方法可动态调整不同类别下各网络的权重,实现类别特异性优化。在此基础上,架构进一步使用CRF中的单变量和双变量势函数优化融合结果,确保空间一致性与边界准确性。为验证ClassWise-CRF的有效性,我们在两个遥感数据集LoveDA和Vaihingen上,使用八种经典与先进分割网络进行了实验。结果表明,该架构显著提升了分割性能:在LoveDA数据集上,验证集mIoU提升1.00%,测试集提升0.68%;在Vaihingen数据集上,验证集提升0.87%,测试集提升0.91%。这些结果充分证明了ClassWise-CRF在遥感图像语义分割中的有效性与通用性。完整代码已开源:https://github.com/zhuqinfeng1999/ClassWise-CRF。
原文摘要 · Abstract (English)
We propose a result-level category-specific fusion architecture called ClassWise-CRF. This architecture employs a two-stage process: first, it selects expert networks that perform well in specific categories from a pool of candidate networks using a greedy algorithm; second, it integrates the segmentation predictions of these selected networks by adaptively weighting their contributions based on their segmentation performance in each category. Inspired by Conditional Random Field (CRF), the ClassWise-CRF architecture treats the segmentation predictions from multiple networks as confidence vector fields. It leverages segmentation metrics (such as Intersection over Union) from the validation set as priors and employs an exponential weighting strategy to fuse the category-specific confidence scores predicted by each network. This fusion method dynamically adjusts the weights of each network for different categories, achieving category-specific optimization. Building on this, the architecture further optimizes the fused results using unary and pairwise potentials in CRF to ensure spatial consistency and boundary accuracy. To validate the effectiveness of ClassWise-CRF, we conducted experiments on two remote sensing datasets, LoveDA and Vaihingen, using eight classic and advanced semantic segmentation networks. The results show that the ClassWise-CRF architecture significantly improves segmentation performance: on the LoveDA dataset, the mean Intersection over Union (mIoU) metric increased by 1.00% on the validation set and by 0.68% on the test set; on the Vaihingen dataset, the mIoU improved by 0.87% on the validation set and by 0.91% on the test set. These results fully demonstrate the effectiveness and generality of the ClassWise-CRF architecture in semantic segmentation of remote sensing images. The full code is available at https://github.com/zhuqinfeng1999/ClassWise-CRF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。