对比两种动态跳层推理方法，发现SWIFT更准更快，且可复现。
A rigor-matched audit of periodic-step layer skipping for efficient llm inference: conflayers versus swift, with a supplemental analysis of trained routing alternatives
- 通过在线搜索动态决定每步执行哪些层，实现高效推理。
- SWIFT在四组实验中三项准确率领先，推理速度普遍快5%-21%。
- 首次公开可复现的严谨评测协议，适合研究高效推理的学者参考。
针对高效大模型推理中的层跳过方法，本文开展了一场严格匹配、三种子的审计，对比两种周期性步骤的在线搜索方法：基于置信度的提前退出基线（ConfLayers）和真正的自推测解码（SWIFT, Xia et al. 2024），以及标准自回归解码，在两个模型规模（Qwen2.5-0.5B 和 Qwen2.5-1.5B）与两个任务（GSM8K 推理与 CNN/DailyMail 摘要生成）上进行评估。在四个实验组中，SWIFT 在准确性上表现最强；而 ConfLayers 在所有场景中均落后，尤其在 1.5B 模型的 GSM8K 任务上差距显著。剔除在线搜索开销后，SWIFT 的实际推理速度在全部四组中均优于 ConfLayers（快 5%-21%），逆转了原始时钟时间排名的判断。ConfLayers 的搜索开销小且稳定（占总成本 1%-2%），而 SWIFT 的开销更大且波动更大（最高达 28.7%）。此外，作为补充分析，我们考察了两种训练式路由方法：LayerRoute（Sikdar, 2026）和 LayerDrop（Fan et al. 2020），其决策粒度较粗。在经验证的完整输入门控、真实全模型基线和真实推理时计算跳过的协议下，两者仅带来小幅加速（1.08-1.33x），但准确率远低于周期性方法，尤其在 1.5B 模型的 GSM8K 上，LayerRoute 准确率近乎崩溃（三次种子平均精确匹配率仅为 0.003）。论文公开完整审计协议，供后续严谨效率比较使用。
原文摘要 · Abstract (English)
Layer-skipping methods for efficient LLM inference decide, at some granularity, which transformer layers to execute for a given input. We present a rigor-matched, three-seed audit of two periodic-step, search-based methods that make this decision online at inference time and re-evaluate it every few generation steps: a confidence-gated early-exit baseline (ConfLayers) and genuine self-speculative decoding (SWIFT, Xia et al. 2024), together with vanilla autoregressive decoding, across two model scales (Qwen2.5-0.5B and Qwen2.5-1.5B) and two tasks (GSM8K reasoning and CNN/DailyMail summarization). SWIFT is the strongest method on accuracy in three of four cells; ConfLayers is dominated everywhere, with particularly large deficits on GSM8K at 1.5B. Once online-search overhead is separated from pure inference cost, SWIFT's true inference speed is faster than ConfLayers's in all four cells (5-21%), reversing the naive wall-clock ranking in three of them. ConfLayers's search overhead is small and stable (1-2% of cost), while SWIFT's is larger and more variable (up to 28.7%). We additionally examine two trained-routing methods, LayerRoute (Sikdar, 2026) and LayerDrop (Fan et al. 2020), as a supplemental analysis because they operate at coarser decision granularities. Under a verified protocol with genuine per-input gating, a genuine full-model baseline, and genuine inference-time compute skipping, both show modest speedups (1.08-1.33x) but accuracy well below the periodic-step methods, including a near-total collapse for LayerRoute on GSM8K at 1.5B (0.003 mean exact-match across three seeds). We release the full audit protocol as a template for rigor-matched efficiency comparisons.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。