arXiv:2512.08048cs.CV2025-12中稿 · TMLR 2026, camera-…

对比空间与频域掩码对持续测试时自适应的影响,发现掩码类型决定性能成败。

Family Matters: A Systematic Study of Spatial vs. Frequency Masking for Continual Test-Time Adaptation

  • 固定策略与损失,仅改变掩码家族(空间/频域)进行系统比较
  • 在ViT上,空间掩码稳定有效,频域掩码会灾难性崩溃
  • 架构与任务匹配决定最优掩码:CNN无差异,细粒度任务中频域掩码可竞争

近期持续测试时自适应(CTTA)方法采用掩码图像建模以应对分布偏移,但多数将掩码家族F视为固定设计,只优化选择策略S,忽视了家族维度的探索。本文通过可控的CTTA框架Mask to Adapt(M2A),固定策略S=随机、标准损失,仅在空间(块、像素)与频域(全带、低带、高带)掩码家族间切换,保持其余组件一致。研究得出关键指导:(1)掩码家族决定适应是否积累有效结构或错误;在块令牌化架构上,空间掩码在长序列中保持稳定表示,而频域掩码会灾难性崩溃;我们通过结构保留机制解释——空间一致性维持宽谱冗余,避免与扰动谱特征完全重叠。(2)最优掩码依赖于架构-任务对齐:在具有重叠感受野的CNN上,掩码差异消失;但在依赖全局线索、大容量的ViT处理细粒度任务时,频域掩码表现竞争力。在混杂的系统级对比中,随机选择表现媲美启发式策略,但此结果仅作参考,不量化策略相对重要性。

原文摘要 · Abstract (English)

Recent continual test-time adaptation (CTTA) methods adopt masked image modeling to stabilize learning under distribution shift, yet each treats its masking family F as a fixed design choice and innovates exclusively along the selection strategy S, leaving the family axis underexplored. We present a systematic empirical study that isolates this axis. Using a controlled CTTA instantiation, Mask to Adapt (M2A), that fixes S=random and standard losses, we vary only F across spatial (patch, pixel) and frequency (all-band, low-band, high-band) families while keeping every other component identical. The study's contributions are the design guidance it extracts for the CTTA settings we evaluated: (1) the masking family determines whether adaptation compounds useful structure or compounds errors, on patch-tokenized architectures, spatial masking accumulates stable representations over long streams while frequency masking collapses catastrophically. We characterize this instability through a structural-preservation account, where spatial coherence maintains the broad-spectrum redundancy needed to avoid terminally overlapping with a corruption's spectral signature; (2) the optimal family depends on architecture-task alignment, on CNNs, whose overlapping receptive fields dilute patch occlusion, the family gap vanishes, whereas on fine-grained tasks with global cues and large-capacity ViTs, frequency masking becomes competitive. In confounded system-level comparisons, where baselines also differ in losses and auxiliary components, M2A's random selection performs comparably to heuristic strategies, though we treat this observation as suggestive context rather than a controlled quantification of S's relative importance.

自适应掩码视觉模型架构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。