不同数据集上,分割模型对特征频谱的敏感性差异显著。
Feature-Spectral Fragility in Segmentation: Dataset Dependence, Architecture-Specific Localization, and Spectral Correlates

- 通过低通滤波干预内部特征,测试模型对频谱成分的依赖性
- 在CVC数据集上,模型性能下降超70%,而ISIC仅轻微下降
- 敏感区域因架构而异,且频谱能量与脆弱性存在反向关联
分割模型的鲁棒性通常通过输入域扰动评估,而对其学习特征表示中频率内容依赖性的理解仍不充分。本文针对三种分割架构(ResNet50-UNet/CNN、VM-UNet/SSM、Swin-UNETR/Transformer),在CVC-ClinicDB和ISIC2018数据集上,采用训练后特征域低通滤波干预,评估其对频谱成分的敏感性。当截止频率rho=0.25时,特征域低通滤波导致CVC数据集上的Dice分数分别下降100%、73.2%和30.9%,而ISIC仅下降9.4%、10.3%和0.6%,跨数据集差异具有统计显著性。单阶段干预显示,敏感性在不同架构中位于特定深度:CNN集中在中后期编码器,SSM则在早期编码器阶段。原生特征频谱测量显示,CVC上高频能量与脆弱性呈负相关,而在ISIC上关系较弱,仅为候选关联。傅里叶增强可提升输入域鲁棒性,但对特征域退化无改善。结果表明,特征频谱鲁棒性强烈依赖数据集、架构特异性,且不同于输入域频谱鲁棒性。
原文摘要 · Abstract (English)
Robustness of segmentation models is commonly assessed through input-domain perturbations, while dependence on frequency content within learned feature representations remains less understood. We probe this dependence using targeted post-training low-pass interventions on internal representations of three segmentation architectures, ResNet50-UNet (CNN), VM-UNet (SSM), and Swin-UNETR (Transformer), across CVC-ClinicDB and ISIC2018, with headline evaluations performed on untouched held-out test sets. At cutoff rho=0.25, feature-domain low-pass filtering causes severe degradation on CVC: Dice drops by 100%, 73.2%, and 30.9% for CNN, SSM, and Transformer, respectively, compared with 9.4%, 10.3%, and 0.6% on ISIC. The cross-dataset difference is statistically significant for every architecture. Single-stage interventions further show that sensitivity is localized at architecture-specific depths: the CNN peaks at a mid/late encoder block, whereas the SSM peaks in an early encoder stage on both datasets. Native feature-domain spectral measurements show an inverse association between high-frequency energy and fragility on CVC; the relationship is only partial on ISIC and is therefore treated as a candidate correlate rather than a proven mechanism. Finally, Fourier augmentation improves robustness to input-space low-pass filtering but leaves feature-domain degradation essentially unchanged. These results show that feature-spectral robustness is strongly dataset-dependent, architecture-specific, and distinct from input-domain spectral robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。