解决视觉Transformer因分块网格相位导致的预测不稳问题
Phase Marginalization for Patch-Grid Instability in Vision Transformers
- 通过后处理对齐不同分块相位的输出，实现稳定预测
- 在城市街景数据集上提升0.31的平均交并比
- 无需训练，适合部署时提升密集预测模型性能
视觉Transformer基于固定分块网格运行，可能导致密集预测中的相位依赖性不稳定：改变分块方式会改变像素可获得的标记证据，尤其在边界附近。本文将分块网格相位视为干扰变量，提出相位边缘化方法，评估结构化分块相位，逆向对齐密集输出，并在原始图像坐标系下聚合。核心变体——K=4的均匀相位边缘化，无需训练，在分割、深度估计和局部匹配任务中均优于标准K=1基线。在受控的城市景观实验中，其在计算量相当条件下，相比通用的四次前向测试时增强（TTA）提升0.31的平均交并比。扩展研究表明，K=4是成本-精度的合理权衡：K=8效果基本不变，而K=16虽小幅增益却显著增加延迟。这些结果表明分块网格相位是可度量的干扰变量，相位边缘化为密集ViT预测提供了简单有效的诊断与后处理基准。
原文摘要 · Abstract (English)
Vision Transformers operate on fixed patch grids, which can introduce phase-dependent instability for dense prediction: changing the patch partition can change the token evidence available to a pixel, especially near boundaries. We formalize patch-grid phase as a nuisance variable and propose Phase Marginalization, a post-hoc marginalization method that evaluates structured patch-grid phases, inverse-aligns dense outputs, and aggregates them in the original image coordinate system. The central variant, Uniform Phase Marginalization with K = 4, is training-free and improves over the canonical K = 1 baseline across measured segmentation, depth, and local matching settings. In a controlled Cityscapes experiment, Uniform Phase Marginalization provides a modest compute-matched advantage over generic shift-based four-forward test-time augmentation (TTA) (+0.31 mean Intersection-over-Union over the strongest tested generic row). A scaling study further shows that K = 4 is a practical cost-accuracy trade-off: K = 8 is essentially unchanged and K = 16 adds little accuracy at much higher latency. These results position patch-grid phase as a measurable nuisance variable and Phase Marginalization as a simple diagnostic and post-hoc marginalization baseline for dense ViT prediction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。