arXiv:2605.04946cs.LGstat.ML2026-05被引 1

批归一化通过重定点几何重塑网络局部分区结构

Training-Time Batch Normalization Reshapes Local Partition Geometry in Piecewise-Affine Networks

论文配图:Training-Time Batch Normalization Reshapes Local Partition Geometry in Piecewise-Affine Networks
图 1 · 摘自论文原文
  • 以批次为中心重定每个神经元的参考超平面
  • 在ReLU等网络中提升局部区域划分精细度
  • 适合研究深层网络几何特性的研究人员

批归一化(BN)是现代深度网络的核心,但其在训练过程中对函数实现的影响仍不如优化优势清晰。本文通过连续分段仿射(CPA)网络中的切换超平面几何和诱导的仿射区域划分,研究训练时的BN作用。给定一个小型批次,我们证明每个神经元的参考超平面通过批次中心定义,而断点切换超平面是平行平移,其偏移量以批量标准化坐标表达,且与原始偏置无关。这给出了切换超平面何时与局部ℓ∞窗口相交的精确判据,并提出基于精确仿射区域计数的局部区域密度函数。在明确的充分条件下,我们证明了BN在ReLU及更一般的分段仿射网络中提升了期望的局部划分细化程度,且该机制可在父仿射区域内沿深度局部传递,前提是上游表示映射为仿射嵌入。这些结果从函数级几何角度揭示了训练时BN作为数据附近的批条件重定点机制。

原文摘要 · Abstract (English)

Batch normalization (BN) is central to modern deep networks, but its effect on the realized function during training remains less understood than its optimization benefits. We study training-time BN in continuous piecewise-affine (CPA) networks through the geometry of switching hyperplanes and the induced affine-region partition. Conditioned on a mini-batch, we show that BN defines for each neuron a reference hyperplane through the batch centroid, and that breakpoint-switching hyperplanes are parallel translates whose offsets are expressed in batch-standardized coordinates and are independent of the raw bias. This yields an exact criterion for when a switching hyperplane intersects a local $\ell_\infty$ window and motivates a local region-density functional based on exact affine-region counts. Under explicit sufficient conditions, we show that BN increases expected local partition refinement in ReLU and more general piecewise-affine networks, and that this mechanism transfers locally through depth inside parent affine regions where the upstream representation map is an affine embedding. These results provide a function-level geometric account of training-time BN as a batch-conditional recentering mechanism near the data.

批归一化几何分析分段仿射

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。