通过低秩跨通道混合提升测试时自适应的可靠性
MixTTA: Low-Rank Cross-Channel Mixing for Reliable Test-Time Adaptation
- 在归一化层引入低秩跨通道变换，实现通道间动态混合
- 在标准与极端场景下均优于主流基线，适应失败率显著降低
- 可无缝集成于现有方法，适合部署中需持续适应的模型
测试时自适应（TTA）方法通常通过更新归一化层的仿射参数来应对分布偏移。然而，逐通道仿射参数仅能进行轴对齐的缩放与平移，无法纠正由分布偏移引起的跨通道结构变化。为此，我们提出 MixTTA，一个轻量级插件模块，在归一化层引入低秩跨通道变换，实现每层的通道间混合。为确保低秩分支仅捕捉跨通道交互，我们设计了解耦投影，强制其与对角仿射路径严格分离；同时采用谱投影，防止非平稳测试流下的秩-1坍塌。MixTTA 可无缝嵌入任意基于归一化层的 TTA 方法。标准与极端测试环境下实验均显示一致性能提升，且在挑战性条件下显著缓解适应失败问题。代码已开源：https://github.com/delta6189/MixTTA。
原文摘要 · Abstract (English)
Test-Time Adaptation (TTA) methods commonly update the affine parameters of normalization layers to adapt deployed models under distribution shifts. However, per-channel affine parameters perform axis-aligned scaling and shifting, making them geometrically incapable of correcting cross-channel structural changes induced by distribution shift. To address this limitation, we propose MixTTA, a lightweight plug-in module that equips normalization layers with a low-rank cross-channel transformation, enabling inter-channel mixing at each layer. To ensure that the low-rank branch captures only cross-channel interactions, we also propose Decoupling Projection that enforces strict separation from the diagonal affine path, along with Spectral Projection that prevents rank-1 collapse under non-stationary test streams. MixTTA can be seamlessly integrated into any existing normalization-based TTA method. Experiments in both standard and wild TTA settings show consistent improvements over strong baselines while mitigating adaptation failure under challenging conditions. The source code is publicly available at https://github.com/delta6189/MixTTA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。