发现大模型推理可分解，实现高效并行计算。
Why Inference in Large Models Becomes Decomposable After Training
- 训练后参数依赖具有局部性，部分结构可被剥离。
- 通过统计准则移除无效依赖，暴露稳定独立子结构。
- 无需改动模型即可实现并行推理，适合部署优化。
大规模AI模型的推理通常在稠密参数矩阵上进行，导致计算成本与系统复杂度随模型规模不可持续增长。这一限制并非源于模型容量不足，而是因将训练后的推理系统视为整体操作，忽视了学习过程中形成的内部结构。我们发现，大规模模型中的梯度更新事件高度局部且选择性强，训练后许多参数依赖在统计上与初始化分布难以区分。因此，训练后的推理系统结构不均一，本质上可分解。基于此，我们提出一种训练后统计判据和结构退火过程，用于消除不支持的依赖关系，揭示出稳定且独立的子结构。本工作建立了训练后、模型无关的推理系统结构视角，实现了无需修改模型功能或接口的结构化、并行推理。
原文摘要 · Abstract (English)
Inference in large-scale AI models is typically performed on dense parameter matrices, leading to inference cost and system complexity that scale unsustainably with model size. This limitation does not arise from insufficient model capacity, but from treating post-training inference systems as monolithic operators while ignoring internal structures formed during learning. We show that gradient update events in large models are highly localized and selective, leaving many parameter dependencies statistically indistinguishable from their initialization distribution after training. As a result, post-training inference systems are structurally non-uniform and inherently decomposable. Based on this observation, we introduce a post-training statistical criterion and a structural annealing procedure that removes unsupported dependencies and reveals stable, independent substructures. This work establishes a post-training, model-agnostic structural view of inference systems and enables structured, parallel inference without modifying model functionality or interfaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。