通过句与层双重提前退出，显著降低大模型推理开销。
Two-dimensional early exit optimisation of LLM inference
- 逐句处理并逐步激活深层，实现层与句维度协同优化。
- 在情感分类任务中提速1.4至2.3倍，复杂任务仍保持稳定。
- 无需重训练，兼容量化剪枝，适合序列处理场景。
我们提出一种二维（2D）提前退出策略，协调大语言模型在分类任务中按层和按句的提前退出。通过逐句增量处理并逐步激活更深层，该方法实现了层与句维度优化的乘积式计算节省。在四个主流LLM（Llama 3.1、Llama 3.2、Gemma、Qwen；参数量3B-8B）上，于三个情感分类数据集进行实验，结果显示，相较于最优层级提前退出，简单任务下额外获得1.4–2.3倍加速，复杂多分类任务中性能平稳下降。微调可减小但无法消除此优势。该方法模型无关，仅需轻量级分类适配器，且与量化、剪枝等效率技术正交。研究表明，当输入结构中语义信息可预测性累积时，2D提前退出表现优异，提示其可能适用于情感分类之外的序列处理任务。
原文摘要 · Abstract (English)
We introduce a two-dimensional (2D) early exit strategy that coordinates layer-wise and sentence-wise exiting for classification tasks in large language models. By processing input incrementally sentence-by-sentence while progressively activating deeper layers, our method achieves multiplicative computational savings that exceed those from optimizing either dimension independently. Experimental evaluation across four state-of-the-art LLMs (Llama 3.1, Llama 3.2, Gemma, Qwen; 3B-8B parameters) on three sentiment classification datasets demonstrates additional speed-ups of 1.4--2.3$\times$ over optimal layer-wise early exit for simpler tasks with vanilla models, with graceful degradation on complex multi-class problems. Fine-tuning reduces but does not eliminate this advantage. The approach is model-agnostic, requires only lightweight classification adapters, and is orthogonal to complementary efficiency methods such as quantization and pruning. Our findings indicate that 2D early exit strategies excel when semantic information accumulates predictably across input structure, suggesting possible applicability to sequence-processing tasks beyond sentiment classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。