TTA在多数情况下提升模型鲁棒性，但对轻微噪声反而有害，需按条件评估其效果。
When Test-Time Adaptation Helps, Harms, or Becomes Inactive: A Condition-Level Study on CIFAR-10-C
- 对比三种TTA方法在15种噪声类型上的表现，分析其适应机制。
- 平均提升12.2–13.3个百分点，但在8.0–9.3%条件下反而低于原始模型。
- 可靠性过滤限制了适应效果，使EATA接近仅调整BN统计量的方法。
测试时自适应（TTA）通过利用无标签测试数据改进模型在分布偏移下的鲁棒性。尽管TENT和EATA等方法在损坏数据上表现良好，但整体准确率可能掩盖适应失败的场景。本文在包含15种腐蚀类型和5个严重等级的CIFAR-10-C基准上，对三种TTA策略——批量归一化统计量自适应（BN-Adapt）、熵最小化自适应（TENT）以及可靠性过滤自适应（EATA的重构实现）——与未自适应源模型进行对照。所有方法均使平均准确率提升12.2–13.3个百分点（Wilcoxon符号秩检验，p < 10⁻¹²）。然而，每种方法在8.0–9.3%的条件下表现劣于源模型，失败集中在低严重度腐蚀（如亮度、雾化、对比度、模糊），此时源模型已接近天花板性能。进一步发现，EATA与无梯度的BN-Adapt基线差异仅0.09个百分点，远小于与TENT的1.08个百分点差异，表明可靠性过滤显著抑制有效适应，使EATA更像一个仅更新批归一化统计量的方法。结果表明，仅依赖整体准确率会掩盖系统性失效模式，推动需在条件层面评估自适应何时有效、无效或有害。
原文摘要 · Abstract (English)
Test-time adaptation (TTA) aims to improve model robustness under distribution shift by adapting a source model using unlabeled test data. Although methods such as TENT and EATA have demonstrated gains on corrupted data, aggregate accuracy can obscure the conditions under which adaptation fails or provides little benefit. We present a controlled comparison of three TTA strategies---BatchNorm-statistics adaptation (BN-Adapt), entropy-minimization adaptation (TENT), and reliability-filtered adaptation (a scoped re-implementation of EATA)---against an unadapted source model on the full CIFAR-10-C benchmark, covering 15 corruption types and 5 severity levels. All three methods improve mean accuracy over the source model by 12.2--13.3 percentage points (Wilcoxon signed-rank $p < 10^{-12}$). However, each method underperforms the source model on 8.0--9.3\% of conditions, with failures concentrated in low-severity corruptions where the source model already performs near ceiling, particularly brightness, fog, contrast, and defocus blur. We further find that EATA closely tracks the gradient-free BN-Adapt baseline, with a mean absolute difference of 0.09 percentage points, compared with 1.08 percentage points relative to TENT. This suggests that reliability filtering can substantially restrict effective adaptation, causing EATA to behave more like a BatchNorm-statistics baseline than an entropy-minimization method. These results show that aggregate accuracy alone can mask systematic TTA failure modes and motivate condition-level evaluation of when adaptation helps, harms, or becomes effectively inactive.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。