ACN通过分组相似数据实现局部归一化,提升图像处理模型性能。
Adaptative Context Normalization: A Boost for Deep Learning in Image Processing
- 按数据特征分组为上下文,每组用统一参数归一化
- 训练时自动学习各上下文归一化参数,收敛更快
- 比BN和MN更快更准,适合图像处理任务
深度神经网络在图像处理中面临层间分布变化的挑战,影响模型收敛与性能。虽然批归一化(BN)等方法已显著改进,但其假设数据服从单一高斯分布,限制了表达能力。混合归一化(MN)引入高斯混合模型(GMM)以建模多成分分布,但需使用期望最大化算法估算参数,计算开销大。为此,本文提出自适应上下文归一化(ACN),引入‘上下文’概念,将具有相似特征的数据分组,同一组内采用相同归一化参数,实现基于上下文的局部表示。各上下文的归一化参数在反向传播中作为可学习权重优化。ACN在速度、收敛性和性能上均优于BN与MN,为图像处理提供了新视角。
原文摘要 · Abstract (English)
Deep Neural network learning for image processing faces major challenges related to changes in distribution across layers, which disrupt model convergence and performance. Activation normalization methods, such as Batch Normalization (BN), have revolutionized this field, but they rely on the simplified assumption that data distribution can be modelled by a single Gaussian distribution. To overcome these limitations, Mixture Normalization (MN) introduced an approach based on a Gaussian Mixture Model (GMM), assuming multiple components to model the data. However, this method entails substantial computational requirements associated with the use of Expectation-Maximization algorithm to estimate parameters of each Gaussian components. To address this issue, we introduce Adaptative Context Normalization (ACN), a novel supervised approach that introduces the concept of "context", which groups together a set of data with similar characteristics. Data belonging to the same context are normalized using the same parameters, enabling local representation based on contexts. For each context, the normalized parameters, as the model weights are learned during the backpropagation phase. ACN not only ensures speed, convergence, and superior performance compared to BN and MN but also presents a fresh perspective that underscores its particular efficacy in the field of image processing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。