让大模型按需跳过或重复层,更省力地推理
Skip a Layer or Loop It? Learning Program-of-Layers in LLMs

- 用可跳过或循环的层组合,动态生成每条输入的推理路径
- 数学推理任务上准确率更高,且平均少执行30%以上的层
- 适合追求高效推理、想挖掘模型隐藏能力的研究者
大语言模型推理通常按固定层数和顺序逐层执行,不具灵活性。本文发现,预训练层可作为模块,在无需重新训练的情况下,通过跳过或循环实现动态的程序化执行(PoLar),为不同输入定制推理路径。多数情况下,更短的执行路径能达到相同甚至更高的准确率,且能修正原模型的错误预测。这表明标准前向传播仅利用了模型潜在推理能力的一小部分。为此,我们提出轻量级的PoLar预测网络,能为每条输入动态生成跳层或重复层的执行方案。在数学推理基准上的实验显示,该方法在减少层数的同时持续提升准确率,且在分布外数据上仍保持优势。
原文摘要 · Abstract (English)
Large language models (LLMs) perform inference by following a fixed depth and order, non-recurrent execution of all layers. We reveal the wide existence of training-free, flexible, dynamic program-of-layers (PoLar), where pretrained layers can be packed as modules and then skipped or looped to form a customized program for each input. For most inputs, substantially shorter program executions can achieve the same or better accuracy, while incorrect predictions of the original LLM can be corrected by alternative programs with fewer layers. These observations indicate that inference admits multiple valid latent computations beyond the standard forward pass. To efficiently achieve PoLar in practice, we propose a lightweight PoLar prediction network, which learns to generate execution programs that dynamically skip or repeat pretrained layers for each input. Experiments on mathematical reasoning benchmarks demonstrate that PoLar consistently improves accuracy over standard inference and prior dynamic-depth methods, often while executing fewer layers, and that these gains persist under out-of-distribution evaluation. Our results suggest that fixed-depth execution captures only a narrow subset of an LLM's latent reasoning capacity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。