提出新型归一化方法，解决图像修复中Transformer训练不稳问题
Analyzing the Training Dynamics of Image Restoration Transformers: A Revisit to Layer Normalization
- 设计i-LN，按输入自适应整体归一化特征
- 使特征幅度从百万量级收敛，通道熵恢复
- 适合图像修复任务，尤其对Transformer有效
本文分析了图像修复（IR）Transformer的训练动态，发现传统层归一化（LN）会导致特征幅度发散至百万量级，并引发通道熵崩溃。研究指出，这源于网络试图绕过与IR任务不匹配的LN约束。针对两个核心错配：1）逐令牌归一化破坏空间相关性；2）输入无关缩放丢弃输入特定统计信息，我们提出面向图像修复的专用层归一化i-LN。该方法全局、自适应地归一化特征并按输入重缩放。理论与实验证明，此设计显著改善训练动态与性能，在多个基准上验证有效。
原文摘要 · Abstract (English)
This work analyzes the training dynamics of Image Restoration (IR) Transformers and uncovers a critical yet overlooked issue: conventional LayerNorm (LN) drives feature magnitudes to diverge to a million scale and collapses channel-wise entropy. We analyze this in the perspective of networks attempting to bypass LN's constraints that conflict with IR tasks. Accordingly, we address two misalignments between LN and IR: 1) per-token normalization disrupts spatial correlations, and 2) input-independent scaling discards input-specific statistics. To address this, we propose Image Restoration Transformer Tailored Layer Normalization i-LN, a simple drop-in replacement that normalizes features holistically and adaptively rescales them per input. We provide theoretical insights and empirical evidence that this simple design effectively leads to both improved training dynamics and thereby improved performance, validated by extensive experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。