让大模型生成更高效，能动态跳过复杂句子的计算步骤。
ADEPT: Adaptive Dynamic Early-Exit Process for Transformers
- 根据每个词的难易程度动态决定是否提前退出计算
- 语言生成效率提升25%，分类任务速度加快4倍
- 适合需要快速响应的大模型应用，如对话系统
大语言模型推理带来巨大计算负担，常需处理数十亿参数。尽管早退出策略能在生成初期减少计算量，但现有方法仅适用于生成首词或预填充阶段，导致跳过的层仍保留键值（KV）缓存，成为后续生成的瓶颈。我们提出 ADEPT（自适应动态早退出机制），可实现预填充和生成阶段的动态早退出。该方法基于词复杂度自适应调整计算，优化效率且不损失性能。ADEPT还通过解耦跳过层间的顺序依赖关系，改进了KV生成过程，使词级早退出更可行。实验表明，ADEPT在语言生成任务中效率最高提升25%，下游分类任务速度提升4倍，性能最高提升45%。
原文摘要 · Abstract (English)
The inference of large language models imposes significant computational workloads, often requiring the processing of billions of parameters. Although early-exit strategies have proven effective in reducing computational demands by halting inference earlier, they apply either to only the first token in the generation phase or at the prompt level in the prefill phase. Thus, the Key-Value (KV) cache for skipped layers remains a bottleneck for subsequent token generation, limiting the benefits of early exit. We introduce ADEPT (Adaptive Dynamic Early-exit Process for Transformers), a novel approach designed to overcome this issue and enable dynamic early exit in both the prefill and generation phases. The proposed adaptive token-level early-exit mechanism adjusts computation dynamically based on token complexity, optimizing efficiency without compromising performance. ADEPT further enhances KV generation procedure by decoupling sequential dependencies in skipped layers, making token-level early exit more practical. Experimental results demonstrate that ADEPT improves efficiency by up to 25% in language generation tasks and achieves a 4x speed-up in downstream classification tasks, with up to a 45% improvement in performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。