解释大模型能力为何突然出现,揭示其源于复杂系统的涌现机制。
Why are LLMs' abilities emergent?
- 从非线性动力学角度分析大模型能力的涌现本质
- 发现能力跃迁源于系统级动态交互,而非单纯参数量增加
- 适合对AI本质和复杂系统感兴趣的读者
大语言模型在生成任务中的惊人成功引发了对其能力本质的根本性疑问——这些能力往往在未明确训练的情况下突然出现。本文通过理论分析与实证观察,探讨深度神经网络(DNNs)的涌现特性,回应当代人工智能发展中“创造却无理解”的认识论难题。研究指出,神经方法依赖非线性、随机过程,使得宏观行为无法从微观神经元活动解析得出。通过对缩放定律、悟道现象及模型能力相变的分析,表明涌现能力源于高度敏感的非线性系统复杂动力学,而非仅由参数规模驱动。研究揭示,当前关于指标、预训练损失阈值和上下文学习的争论,忽略了DNN中涌现的本质本体论特征。作者认为,这些系统展现出与自然复杂现象类似的真正涌现属性,即系统能力由简单组件间的协同作用产生,且不可还原为单个组件的行为。论文结论强调,理解大模型能力需将其视为受普遍涌现规律支配的新一类复杂动力系统,类似于物理、化学与生物学中的规律。这一视角将关注点从现象学定义转向理解内部动态转化,使系统获得超越其组成部分的能力。
原文摘要 · Abstract (English)
The remarkable success of Large Language Models (LLMs) in generative tasks has raised fundamental questions about the nature of their acquired capabilities, which often appear to emerge unexpectedly without explicit training. This paper examines the emergent properties of Deep Neural Networks (DNNs) through both theoretical analysis and empirical observation, addressing the epistemological challenge of "creation without understanding" that characterises contemporary AI development. We explore how the neural approach's reliance on nonlinear, stochastic processes fundamentally differs from symbolic computational paradigms, creating systems whose macro-level behaviours cannot be analytically derived from micro-level neuron activities. Through analysis of scaling laws, grokking phenomena, and phase transitions in model capabilities, I demonstrate that emergent abilities arise from the complex dynamics of highly sensitive nonlinear systems rather than simply from parameter scaling alone. My investigation reveals that current debates over metrics, pre-training loss thresholds, and in-context learning miss the fundamental ontological nature of emergence in DNNs. I argue that these systems exhibit genuine emergent properties analogous to those found in other complex natural phenomena, where systemic capabilities emerge from cooperative interactions among simple components without being reducible to their individual behaviours. The paper concludes that understanding LLM capabilities requires recognising DNNs as a new domain of complex dynamical systems governed by universal principles of emergence, similar to those operating in physics, chemistry, and biology. This perspective shifts the focus from purely phenomenological definitions of emergence to understanding the internal dynamic transformations that enable these systems to acquire capabilities that transcend their individual components.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。