揭示大模型后训练时参数结构的统一变化规律。
Understanding Post-Training Structural Changes in Large Language Models
- 通过奇异值分解分析后训练中线性层的结构变化。
- 发现奇异值均匀缩放与奇异向量一致旋转的规律。
- 为理解模型参数演化提供可解释的新视角，适合研究者参考。
后训练显著改变大型语言模型（LLM）的行为，但其对内部参数空间的影响仍不清晰。本文对预训练LLM中的主要线性层进行系统性奇异值分解（SVD）分析，聚焦指令微调和长链思维（Long-CoT）蒸馏两种常用后训练方法。结果揭示出两个意外且稳健的结构变化：（1）各层奇异值呈现近似均匀的几何缩放；（2）每矩阵的左右奇异向量均经历高度一致的正交变换。基于此，我们提出一个简洁有效的框架，描述LLM参数的协同动态，阐明后训练依赖预训练所建立的基础能力。实验表明，奇异值缩放支撑后训练的温度调控机制，而奇异向量的协同旋转编码了关键语义对齐。这些发现挑战了参数空间为黑箱的传统认知，首次揭示模型训练过程中参数演化的明确规律，为深入研究模型参数变化提供了新视角。
原文摘要 · Abstract (English)
Post-training fundamentally alters the behavior of large language models (LLMs), yet its impact on the internal parameter space remains poorly understood. In this work, we conduct a systematic singular value decomposition (SVD) analysis of principal linear layers in pretrained LLMs, focusing on two widely adopted post-training methods: instruction tuning and long-chain-of-thought (Long-CoT) distillation. Our analysis reveals two unexpected and robust structural changes: (1) a near-uniform geometric scaling of singular values across layers; and (2) highly consistent orthogonal transformations are applied to the left and right singular vectors of each matrix. Based on these findings, We propose a simple yet effective framework to describe the coordinated dynamics of parameters in LLMs, which elucidates why post-training inherently relies on the foundational capabilities developed during pre-training. Further experiments demonstrate that singular value scaling underpins the temperature-controlled regulatory mechanisms of post-training, while the coordinated rotation of singular vectors encodes the essential semantic alignment. These results challenge the prevailing view of the parameter space in large models as a black box, uncovering the first clear regularities in how parameters evolve during training, and providing a new perspective for deeper investigation into model parameter changes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。