发现Transformer微调时梯度被抑制,提出在关键层注入轻量适配器解决。
Attention Saturation and Gradient Suppression at Inflection Layers: Diagnosing and Mitigating Bottlenecks in Transformer Adaptation
- 通过注意力熵与梯度分析定位梯度衰减的关键层。
- 在关键层插入LoRA适配器可恢复反向信号,提升微调效果。
- 适用于需要高效微调的场景,尤其对过拟合或欠拟合初始化有指导意义。
预训练Transformer在微调时常表现出对源域模式的过度自信,难以形成新目标域模式。本文通过交叉熵与softmax分析,揭示输出饱和导致梯度抑制的机制,发现梯度抑制限制了低层特征重构,仅允许高层特征重组。为此,提出一套分层诊断指标:注意力熵(饱和代理)、激活梯度范数、参数梯度范数及共享主成分分析下的Delta-CKA,用于识别同时具备低注意力熵与陡峭梯度衰减的临界层。基于此,提出先诊断后轻量注入的微调策略:仅在临界层插入LoRA适配器,以最小参数开销恢复被抑制的反向信号。在BERT-base从SST-2迁移至Rotten Tomatoes的任务中,实验显示过拟合初始化下临界层注入有益,而欠拟合初始化则性能下降。当基础特征较强时,解封临界层可促进高层组合式适应;当基础特征较弱时,则需全路径解封以实现低层重建,该结论得到逐层激活梯度与Delta-CKA动态联合分析的支持。
原文摘要 · Abstract (English)
Pre-trained Transformers often exhibit over-confidence in source patterns and difficulty in forming new target-domain patterns during fine-tuning. We formalize the mechanism of output saturation leading to gradient suppression through standard cross-entropy and softmax analysis, showing that gradient suppression at inflection layers confines adaptation to high-level recombination of existing features while preventing low-level reconstruction. We introduce a set of layer-wise diagnostic metrics -- attention entropy (saturation proxy), activation gradient norm, parameter gradient norm, and Delta-CKA under a shared PCA basis -- to identify inflection layers characterized by both low attention entropy and steep gradient decay. Building on these findings, we propose a diagnose-first, inject-light fine-tuning strategy: selectively inserting LoRA adapters at inflection layers to restore suppressed backward signals with minimal parameter overhead. Experiments on BERT-base transfer from SST-2 to Rotten Tomatoes under under-trained and over-trained source regimes reveal that over-trained initialization benefits from inflection-layer LoRA injection, while under-trained initialization suffers performance degradation. When base features are strong, unblocking inflection layers facilitates high-level compositional adaptation; when base features are weak, full-pathway unblocking is required for low-level reconstruction, as supported by joint analysis of layer-wise activation gradients and Delta-CKA dynamics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。