筛选更平坦的词元数据，让推测解码训练快一倍以上
Flatter Tokens are More Valuable for Speculative Draft Model Training
- 按目标模型输出分布平坦度筛选训练样本
- 仅用50%数据实现2倍以上训练加速
- 适合想提升推理训练效率的研究者
推测解码（SD）是加速大语言模型推理的关键技术，但通常需要在大规模数据集上训练草稿模型。本文从数据视角出发，发现并非所有训练样本对SD接受率贡献相同。理论分析与实证表明，来自目标模型预测分布更平坦的词元比分布尖锐的更具价值。基于此提出‘平坦度’新指标，设计样本级平坦度数据蒸馏（SFDD）方法，仅保留最优质样本。在EAGLE框架上的实验显示，SFDD仅用50%数据即可实现超过2×的训练加速，最终模型推理速度提升仍保持在全数据基线的4%以内。该工作提出了高效的数据驱动方法，显著提升推测解码训练效率。代码已开源。
原文摘要 · Abstract (English)
Speculative Decoding (SD) is a key technique for accelerating Large Language Model (LLM) inference, but it typically requires training a draft model on a large dataset. We approach this problem from a data-centric perspective, finding that not all training samples contribute equally to the SD acceptance rate. Specifically, our theoretical analysis and empirical validation reveals that tokens inducing flatter predictive distributions from the target model are more valuable than those yielding sharply peaked distributions. Based on this insight, we propose flatness, a new metric to quantify this property, and develop the Sample-level-flatness-based Dataset Distillation (SFDD) approach, which filters the training data to retain only the most valuable samples. Experiments on the EAGLE framework demonstrate that SFDD can achieve over 2$\times$ training speedup using only 50% of the data, while keeping the final model's inference speedup within 4% of the full-dataset baseline. This work introduces an effective, data-centric approach that substantially improves the training efficiency for Speculative Decoding. Our code is available at https://github.com/fjm9933/Flatness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。