MoE专家缓存评估存在三大陷阱，影响结论可靠性。
Reproducible Evaluation of MoE Expert Caching: Replay Semantics, Workload Contamination, and Operating Regimes
- 通过模拟发现缓存策略评估受重放语义、工作负载污染和运行模式影响
- 现有缓存策略距最优仍差44%-46%，其中84%以上由未来使用预测决定
- 基于因果预测的淘汰机制可缩小11.4%差距，优于传统算法
Mixture-of-Experts (MoE) 模型已超出加速器内存容量，将专家权重卸载至主机内存已成为标准。这使得专家缓存管理成为关键优化点：更高命中率可降低每标记的专家通信量。然而评估这一问题存在测量脆弱性。在三个 MoE 模型（40、64、128 个专家）上，基于事件原子的追踪模拟揭示了三个会改变结论而非仅数值的评估维度。重放语义：在融合事件流量协议下，不一致的逐访问重放使基于近期性的策略命中率虚高27-29%，而基于频率和静态策略仅变化4%以内，导致策略排名反转。工作负载污染：使用单一指令模板生成的探测集产生完全相同的生成前缀；采用匹配对渲染干预后，早期窗口效应测量值变动19.4-31.9个百分点，并逆转了哪些工作负载最缓存友好。运行模式：归一化缺失率无法跨模型迁移，必须报告每步专家并集与每层容量之比——但仅改变相同事件流的时间顺序，离线最优差距即从44.9%降至30.8%，因此不足为据。修正后，在13种冻结工作负载组合中，离线最优差距稳定在44.2-45.9%。强制准入预言机表明其中84.3-96.6%源于知晓未来使用最远的驻留专家。基于因果未来使用预测的淘汰规则可恢复11.4%差距；其在3.4%情况下选出最优受害者，优于随机驻留块的2.4%及LRU、LFRU的20.6-22.1%。我们观点明确：在当前评估设置下，离线最优差距显著夸大了代表性轻量级因果机制可恢复的收益。
原文摘要 · Abstract (English)
Mixture-of-Experts (MoE) models have outgrown accelerator memory, and offloading expert weights to host memory is now standard. This makes expert cache management an attractive lever: a policy that raised the hit rate would cut expert traffic per token. Evaluating that is a measurement problem, and we find the measurement fragile. With a trace-driven, event-atomic simulator over three MoE models (40, 64, 128 experts), we isolate three evaluation axes that change conclusions, not just numbers. Replay semantics: under a fused-event traffic contract, an inconsistent per-access replay inflates recency-based policies by 27-29% while leaving frequency-based and static ones within 4%, inverting the policy ranking. Workload contamination: probe sets using one instruction template per category produce verbatim-identical generation prefixes; a matched-pair rendering intervention moves the measured early-window effect by 19.4-31.9 points and reverses which workloads look most cache-friendly. Operating regimes: normalized miss fractions do not transfer across models, so the per-step expert union relative to per-layer capacity must be reported -- yet permuting only the temporal order of an identical event stream moves the offline-optimal gap from 44.9% to 30.8%, so it is not sufficient. Corrected, a stable gap to the offline optimum remains (44.2-45.9% over 13 frozen workload compositions). A forced-admission oracle attributes 84.3-96.6% of it to knowing which resident expert is used furthest in the future. A causal next-use predictor, used as an eviction rule, recovers -11.4% of the gap; it picks an optimal victim 3.4% of the time, against 2.4% for a random resident block and 20.6-22.1% for LRU and LFRU. Our position is narrow: in our evaluated settings a large offline-optimal gap substantially overstates the gains recovered by representative lightweight causal mechanisms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。