根据输入动态跳过冗余层，加速大模型推理
DASH: Input-Aware Dynamic Layer Skipping for Efficient LLM Inference with Markov Decision Policies
- 将跳层决策建模为马尔可夫决策过程，实现细粒度动态计算
- 在多个基准上实现显著加速，性能优于现有方法
- 适合对延迟敏感的应用场景，如实时对话系统
大语言模型在众多自然语言任务中取得了卓越表现，但其高昂的推理成本成为实际部署的主要障碍，尤其在对延迟敏感的场景中。为此，我们提出DASH，一种基于输入特征自适应选择计算路径的动态层跳过框架。我们将跳层过程建模为马尔可夫决策过程（MDP），基于中间表示实现细粒度的逐标记决策。为缓解跳层带来的性能下降，引入轻量级补偿机制，在决策过程中注入差异奖励。此外，设计异步执行策略，将层计算与策略评估重叠，以最小化运行时开销。在多种LLM架构和NLP基准上的实验表明，该方法在保持竞争力任务性能的同时，实现了显著的推理加速，优于现有方法。
原文摘要 · Abstract (English)
Large language models (LLMs) have achieved remarkable performance across a wide range of NLP tasks. However, their substantial inference cost poses a major barrier to real-world deployment, especially in latency-sensitive scenarios. To address this challenge, we propose \textbf{DASH}, an adaptive layer-skipping framework that dynamically selects computation paths conditioned on input characteristics. We model the skipping process as a Markov Decision Process (MDP), enabling fine-grained token-level decisions based on intermediate representations. To mitigate potential performance degradation caused by skipping, we introduce a lightweight compensation mechanism that injects differential rewards into the decision process. Furthermore, we design an asynchronous execution strategy that overlaps layer computation with policy evaluation to minimize runtime overhead. Experiments on multiple LLM architectures and NLP benchmarks show that our method achieves significant inference acceleration while maintaining competitive task performance, outperforming existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。