低层特征易受域偏移影响，导致少样本分割性能先升后降。
The Devil is in Low-Level Features for Cross-Domain Few-Shot Segmentation
- 通过平坦化低层特征的损失曲面，缓解域偏移带来的训练不稳。
- 在1-shot和5-shot下分别提升3.71%和5.34%的平均MIoU。
- 适合跨域少样本分割场景，尤其目标域与源域差异大时。
跨域少样本分割（CDFSS）旨在将大规模源域数据上学习到的像素级分割能力迁移到下游目标域，仅需每类少量标注图像。本文聚焦于一个长期存在但未解决的现象：当目标域与源域差异较大时，分割性能在训练初期即达峰值，随后随源域训练持续下降。我们深入分析发现，低层特征对域偏移极为敏感，导致源域训练过程中损失曲面更陡峭，这是造成性能下降的根源。基于此，我们提出一种包含两个即插即用模块的方法：一是在源域训练中通过新提出的尖锐度感知最小化方法平坦低层特征的损失曲面；二是在目标域测试阶段，利用基于低层特征的校准直接补充目标域信息。在四个目标数据集上的大量实验验证了该思路的有效性，所提方法在1-shot和5-shot场景下分别显著优于当前最优方法3.71%和5.34%的平均MIoU。
原文摘要 · Abstract (English)
Cross-Domain Few-Shot Segmentation (CDFSS) is proposed to transfer the pixel-level segmentation capabilities learned from large-scale source-domain datasets to downstream target-domain datasets, with only a few annotated images per class. In this paper, we focus on a well-observed but unresolved phenomenon in CDFSS: for target domains, particularly those distant from the source domain, segmentation performance peaks at the very early epochs, and declines sharply as the source-domain training proceeds. We delve into this phenomenon for an interpretation: low-level features are vulnerable to domain shifts, leading to sharper loss landscapes during the source-domain training, which is the devil of CDFSS. Based on this phenomenon and interpretation, we further propose a method that includes two plug-and-play modules: one to flatten the loss landscapes for low-level features during source-domain training as a novel sharpness-aware minimization method, and the other to directly supplement target-domain information to the model during target-domain testing by low-level-based calibration. Extensive experiments on four target datasets validate our rationale and demonstrate that our method surpasses the state-of-the-art method in CDFSS signifcantly by 3.71% and 5.34% average MIoU in 1-shot and 5-shot scenarios, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。