arXiv:2601.02907cs.CL2026-01综述被引 7

系统梳理大模型理论机制,打破黑箱困局。

Beyond the Black Box: A Survey on the Theory and Mechanism of Large Language Models

  • 按生命周期划分六阶段,统一研究框架。
  • 剖析数据混合、模型架构、对齐优化等理论问题。
  • 适合关注模型原理与科学化的研究者阅读。

大语言模型的迅速发展推动了人工智能范式的深刻变革,带来了巨大的工程成功,但其理论理解仍相对滞后,导致系统常被视为“黑箱”。为解决这一理论碎片化问题,本文提出基于生命周期的统一分类体系,将研究分为数据准备、模型准备、训练、对齐、推理和评估六个阶段。在此框架下,系统回顾了驱动大模型性能的基础理论与内部机制,分析了数据混合的数学依据、不同架构的表征极限及对齐算法的优化动态等核心理论问题。超越现有实践,指出合成数据自提升的理论边界、安全保证的数学限制以及涌现智能的机制根源等前沿挑战。通过连接经验观察与严谨科学探究,本文为大模型研发从工程经验转向科学范式提供了结构化路径。

原文摘要 · Abstract (English)

The rapid emergence of Large Language Models (LLMs) has precipitated a profound paradigm shift in Artificial Intelligence, delivering monumental engineering successes that increasingly impact modern society. However, a critical paradox persists within the current field: despite the empirical efficacy, our theoretical understanding of LLMs remains disproportionately nascent, forcing these systems to be treated largely as ``black boxes''. To address this theoretical fragmentation, this survey proposes a unified lifecycle-based taxonomy that organizes the research landscape into six distinct stages: Data Preparation, Model Preparation, Training, Alignment, Inference, and Evaluation. Within this framework, we provide a systematic review of the foundational theories and internal mechanisms driving LLM performance. Specifically, we analyze core theoretical issues such as the mathematical justification for data mixtures, the representational limits of various architectures, and the optimization dynamics of alignment algorithms. Moving beyond current best practices, we identify critical frontier challenges, including the theoretical limits of synthetic data self-improvement, the mathematical bounds of safety guarantees, and the mechanistic origins of emergent intelligence. By connecting empirical observations with rigorous scientific inquiry, this work provides a structured roadmap for transitioning LLM development from engineering heuristics toward a principled scientific discipline.

大模型理论机制黑箱问题综述

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。