通过核心扩展路由与统一调度，加速多模态模型推理。
Accelerating Unified Multimodal Models with Core-Expansion Routing and Unified Computation Scheduling
- 设计核心-扩展路由机制，动态分配计算资源。
- 保持98.03%理解性能，推理速度提升1.93倍。
- 适合追求高效多模态推理的开发者和研究者。
统一多模态模型同时支持理解与生成任务，但存在跨标记、层及生成时间步的大量冗余计算。通过标记重要性探测，我们发现一种非对称的核心-扩展结构：理解任务具有稳定的高重要性成分，而生成任务虽共享该成分，但需依赖进度进行修正。为此，我们提出CE-Router，采用任务共享的核心评分器与进度条件生成扩展，通过生成分解与跨任务核心对齐进行优化。推理时，CE-Router压缩标记计算量，并提供学习到的路由信号，由统一计算调度协调层跳过、前馈网络剪枝、扩散头缓存复用及去噪步骤提前退出。在两种代表性统一多模态模型架构上实验表明，任务间均实现质量-效率一致提升：理解性能保留98.03%，端到端推理速度提升1.93倍。
原文摘要 · Abstract (English)
Unified multimodal models jointly support understanding and generation, but incur substantial redundant computation across tokens, layers, and generation timesteps. Through token-importance probing, we identify an asymmetric core-expansion structure: understanding exhibits a stable importance component, while generation largely shares this component but requires progress-dependent corrections. We therefore propose CE-Router, which uses a task-shared core scorer and progress-conditioned generation expansions, optimized through generation decomposition and cross-task core alignment. At inference, CE-Router compacts token computation and supplies a learned routing signal to Unified Computation Scheduling, which coordinates layer skipping, FFN pruning, diffusion-head cache reuse, and denoising-step early exit. Experiments on two representative UMM architectures demonstrate consistent quality--efficiency improvements across both tasks, retaining 98.03\% of dense understanding performance with a 1.93$\times$ end-to-end inference speedup.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。