提出轻量级跳过机制，让大模型推理更快更省参数。
Normalized Narrow Jump To Conclusions: Normalized Narrow Shortcuts for Parameter Efficient Early Exit Transformer Prediction
- 用压缩的线性变换替代传统跳过方法，参数量减少97%以上。
- 在GPT-2-XL等模型上，早期阶段精度稳定超越无变换跳过。
- 适合追求高效推理的大模型部署场景。
随着大型基于Transformer的语言模型规模和成本持续增长，近期研究关注通过将早期隐藏表示快速映射到最终表示来降低推理开销。特别是，对预训练Transformer使用早期层上的线性变换进行跳过已被证明可提升早期推理精度。然而，对于大型语言模型，这种做法仍存在计算负担。本文提出Narrow Jump to Conclusions（NJTC）与归一化版本N-NJTC，作为标准线性跳过的参数高效替代方案，使跳过参数数量减少超过97%。实验表明，N-NJTC在GPT-2-XL、Phi3-Mini和Llama2-7B模型上均能可靠超越身份跳过（Identity shortcuts），并在所有Transformer层级别保持稳定精度，验证了更高效跳过策略的可行性。
原文摘要 · Abstract (English)
With the size and cost of large transformer-based language models growing, recently, there has been interest in shortcut casting of early transformer hidden-representations to final-representations for cheaper model inference. In particular, shortcutting pre-trained transformers with linear transformations over early layers has been shown to improve precision in early inference. However, for large language models, even this becomes computationally expensive. In this work, we propose Narrow Jump to Conclusions (NJTC) and Normalized Narrow Jump to Conclusions (N-NJTC) - parameter efficient alternatives to standard linear shortcutting that reduces shortcut parameter count by over 97%. We show that N-NJTC reliably outperforms Identity shortcuts at early stages and offers stable precision from all transformer block levels for GPT-2-XL, Phi3-Mini and Llama2-7B transformer models, demonstrating the viability of more parameter efficient short-cutting approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。