用一个通用跳跃结构，30倍减少参数开销，实现高效早退出。
One Jump Is All You Need: Short-Cutting Transformers for Early Exit Prediction with One Jump to Fit All Exit Levels
- 设计统一低秩跳跃结构，替代每个层独立跳转
- 在多模型上保持稳定精度，参数量降低超30倍
- 适合需要低延迟推理的大模型部署场景
为降低大语言模型推理的时间与计算成本，研究者提出参数高效的低秩早退出机制，通过将变压器隐藏表示短接到最终表示。现有方法在每个中间层都维护独立的低秩跳转路径，导致参数开销高。本文提出一种单一的‘一跳通适’（OJFA）低秩跳转结构，在推理阶段实现超过30倍的跳转参数压缩。尽管参数极简，其性能仍接近多路径跳转方案，并在GPT2-XL、Phi3-Mini和Llama2-7B模型上均实现各层级稳定的预测精度。
原文摘要 · Abstract (English)
To reduce the time and computational costs of inference of large language models, there has been interest in parameter-efficient low-rank early-exit casting of transformer hidden-representations to final-representations. Such low-rank short-cutting has been shown to outperform identity shortcuts at early model stages while offering parameter-efficiency in shortcut jumps. However, current low-rank methods maintain a separate early-exit shortcut jump to final-representations for each transformer intermediate block-level during inference. In this work, we propose selection of a single One-Jump-Fits-All (OJFA) low-rank shortcut that offers over a 30x reduction in shortcut parameter costs during inference. We show that despite this extreme reduction, our OJFA choice largely matches the performance of maintaining multiple shortcut jumps during inference and offers stable precision from all transformer block-levels for GPT2-XL, Phi3-Mini and Llama2-7B transformer models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。