优化混合滑窗注意力模型推理，显著降低显存与计算开销。
Full-Pipeline Inference Optimization for MiMo-V2.5 Series: Pushing Hybrid SWA Efficiency to the Limit
- 通过分层预取和专用缓存策略，实现线性显存增长的滑窗注意力。
- 构建高吞吐分布式缓存系统，支持多模态输入高效推理。
- 适合大规模多模态大模型部署，工程落地性强。
本文针对MiMo-V2.5系列模型，提出端到端推理优化方案，融合混合滑窗注意力（Hybrid SWA）、稀疏专家混合（MoE）与多模态编码器。尽管Hybrid SWA理论上可大幅降低注意力计算与键值缓存（KVCache）存储，但实际生产中需大量工程优化。我们通过分层预取、面向SWA的前缀缓存树与专用放置策略，实现严格$O(W)$的SWA存储并保持高命中率。进一步构建了基于RDMA优化网络的GCache分布式缓存系统，并开发缓存亲和性路由机制，在保障负载均衡的同时减少冗余计算。同时优化多模态输入处理，包括GPU图像预处理、并行视频解码与跨模态缓存共享。上述优化构成首个在生产环境中高效支撑混合滑窗注意力+MoE+多模态复合架构的大规模大模型服务系统。
原文摘要 · Abstract (English)
We present a full-pipeline inference optimization for the MiMo-V2.5 model family, which combines Hybrid Sliding Window Attention (Hybrid SWA), sparse Mixture-of-Experts (MoE), and multimodal encoders. While Hybrid SWA can ideally reduce both attention compute and KVCache storage significantly compared to Full Attention, realizing these gains in production requires substantial engineering effort. We systematically optimize the KVCache system with layerwise prefetch, SWA-aware prefix cache trees, and specialized placement strategies, achieving strict $O(W)$ SWA storage and high cache hit rates. We further build GCache, a high-performance distributed cache infrastructure with RDMA-optimized networking, and develop a KVCache-affinity router to reduce computation while preserving load balancing. We also optimize for multimodal inputs, including GPU image preprocessing, parallel video decoding, and multimodal cache sharing. Together, these optimizations constitute the first large-scale LLM serving system in production that efficiently covers the Hybrid SWA + MoE + multimodal composite architecture.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。