提出高阶渐近理论，提升批归一化在分布漂移下的测试时自适应性能。
Higher-Order Asymptotics of Test-Time Adaptation for Batch Normalization Statistics
- 结合埃奇沃斯展开与鞍点逼近，构建批归一化统计量的高阶渐近框架。
- 推导出最小化均方误差的最优加权参数，量化偏差-方差-偏度权衡关系。
- 理论揭示高阶修正与鲁棒一步更新如何提升模型可靠性，适合研究者参考。
本研究针对分布漂移下批归一化（Batch Normalization, BN）统计量的测试时自适应（Test-Time Adaptation, TTA），建立了一套高阶渐近理论框架。通过融合经典的埃奇沃斯展开（Edgeworth expansion）与鞍点逼近（saddlepoint approximation）技术，并引入新颖的一步M-估计视角，分析了训练与测试分布间的统计差异。推导出归一化后BN均值差的埃奇沃斯展开，获得最小化适应统计量均方误差的最优加权参数。将BN TTA重视为一步M-估计器，得出包含偏度等高阶矩的局部高阶渐近正态性结果。进一步量化了适应过程中偏差、方差与偏度之间的权衡，并建立了模型风险的通用泛化界。精炼的鞍点逼近方法提供了对BN TTA统计量密度和尾概率的统一高精度估计。这些理论洞见全面揭示了高阶校正与鲁棒一步更新如何增强BN层在数据分布变化下的可靠性与性能。
原文摘要 · Abstract (English)
This study develops a higher-order asymptotic framework for test-time adaptation (TTA) of Batch Normalization (BN) statistics under distribution shift by integrating classical Edgeworth expansion and saddlepoint approximation techniques with a novel one-step M-estimation perspective. By analyzing the statistical discrepancy between training and test distributions, we derive an Edgeworth expansion for the normalized difference in BN means and obtain an optimal weighting parameter that minimizes the mean-squared error of the adapted statistic. Reinterpreting BN TTA as a one-step M-estimator allows us to derive higher-order local asymptotic normality results, which incorporate skewness and other higher moments into the estimator's behavior. Moreover, we quantify the trade-offs among bias, variance, and skewness in the adaptation process and establish a corresponding generalization bound on the model risk. The refined saddlepoint approximations further deliver uniformly accurate density and tail probability estimates for the BN TTA statistic. These theoretical insights provide a comprehensive understanding of how higher-order corrections and robust one-step updating can enhance the reliability and performance of BN layers in adapting to changing data distributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。