提出稳定训练方法,让ViT在材料分割上达到新SOTA,揭示数据划分陷阱。
Revitalizing Dense Material Segmentation: Stabilized Vision Transformers and the Generalization Paradox

- 用高保真对齐、熵正则化等策略稳定ViT训练,解决纹理区域梯度不稳问题。
- 在原始数据集上实现0.4572 mIoU,显著超越此前卷积模型基线。
- 发现数据重划分导致分布同质化,反而损害真实场景泛化能力,适合关注物理感知的团队。
材料分割要求像素级分类物理表面属性,需超越对象中心解析的物化理解。尽管苹果公司推出了严谨的密集材料分割(Apple-DMS)数据集,该基准却因数据衰减与停滞,逐渐被几何偏好型基础模型掩盖。本文复兴Apple-DMS基准,建立现代视觉变换器(Vision Transformer)基线。我们系统评估SegFormer与Mask2Former架构,发现标准训练在无定形纹理场中因高方差梯度而失效。为此,提出包含高保真对齐、查询熵正则化及领域特定物理合规增强的数据增强方案。优化后的SegFormer-B5在原数据集划分下取得0.4572 mIoU的新SOTA,远超原有卷积基线。此外,我们发现关键“泛化悖论”:将数据重划分为80/10/10的富数据集后,指标升至0.5276 mIoU,但专家定性分析表明其引发分布同质化,严重削弱真实世界、分布外表现。通过发布恢复后的数据索引与鲁棒训练框架,本文强调材料感知尚未解决,呼吁社区采用原始严格划分以推动真正物理驱动的人工智能进展。
原文摘要 · Abstract (English)
Material segmentation, the pixel-wise classification of physical surface properties, remains a challenging problem in computer vision, requiring physicochemical understanding distinct from object-centric parsing. Despite the introduction of the rigorous Apple Dense Material Segmentation (DMS) dataset, the benchmark has suffered from attrition and stagnation, increasingly overshadowed by geometry-biased foundation models. In this paper, we revive the Apple-DMS benchmark to establish a modern Vision Transformer baseline. We conduct an exhaustive evaluation of SegFormer and Mask2Former architectures, revealing that standard training paradigms fail on amorphous texture fields due to high-variance gradients. To address this, we introduce a stabilized training recipe featuring High-Fidelity Logit Projection, Query Entropy Regularization, and a domain-specific, physics-compliant augmentation pipeline. Our optimized SegFormer-B5 achieves a new State-of-the-Art (SOTA) of 0.4572 mIoU on the original dataset split, significantly surpassing the prior convolutional baseline. Furthermore, we identify a critical "Generalization Paradox": while re-partitioning the dataset into a data-rich 80/10/10 split inflates the metric to 0.5276 mIoU, expert qualitative analysis reveals this induces distributional homogenization, severely degrading real-world, out-of-distribution performance. By releasing our recovered dataset index and robust training framework, we demonstrate that material perception is far from solved and urge the community to leverage the rigorous original split to drive genuine progress in physically grounded artificial intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。