arXiv:2607.14557cs.AI2026-07中稿 · ACM Multimedia被引 7

发现扩散模型首步即可预判输出长度,一键截断冗余计算

Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncation

论文配图:Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncation
图 1 · 摘自论文原文
  • 利用首步MLP激活稀疏性变化识别真实输出边界
  • 实现31倍吞吐加速,且复杂视觉任务准确率提升
  • 无需训练、即插即用,适合部署高并发多模态系统

扩散多模态大模型(DMLLM)在多模态推理中表现优异,但其推理效率受制于固定长度生成机制。由于输出长度未知,序列需填充至预设最大长度,导致大量冗余计算浪费在无意义的[EOS]标记上。本文发现,DMLLM在首个去噪步骤中便通过MLP激活稀疏性的显著变化,隐式揭示了有效语义边界。基于此,提出Seer框架——一种无需训练的方法,采用信噪比(SNR)准则检测该边界,并对后续所有计算进行一次性截断。为保障批量服务中的性能,Seer引入混合执行策略,在支持动态序列长度的同时最大化吞吐量。实验表明,Seer有效消除填充浪费,吞吐量最高提升约31倍;在9个基准测试中保持整体性能,且在复杂视觉任务中因抑制噪声泄漏而提升准确率(如DocVQA得分从63.52提升至63.66),提供了一种高效、即插即用的DMLLM加速方案。

原文摘要 · Abstract (English)

Diffusion Multimodal Large Language Models (DMLLMs) are highly effective for multimodal reasoning, yet their inference efficiency is significantly hindered by fixed-length generation constraints. Since the actual output length is unknown, output sequences are padded to a predefined maximum length, resulting in substantial redundant computation over unnecessary [EOS] tokens. In this work, we discover that DMLLMs implicitly reveal their valid semantic boundary at the very first denoising step through a distinct shift in MLP activation sparsity. Leveraging this observation, we propose Seer, a training-free framework that detects this boundary using a Signal-to-Noise Ratio (SNR)-based criterion and performs one-shot truncation of the redundant suffix for all subsequent computations. To preserve these theoretical gains during batched serving, Seer incorporates a hybrid execution strategy that maximizes throughput while seamlessly accommodating dynamic sequence lengths. Experimental results demonstrate that Seer effectively eliminates padding waste, accelerating throughput by up to $\sim$31$\times$. Across 9 benchmarks, Seer robustly maintains overall performance and even improves accuracy on complex visual tasks by mitigating noise leakage (e.g., DocVQA score increases from 63.52 to 63.66), offering a highly efficient, plug-and-play solution for DMLLM acceleration.

扩散模型多模态推理加速稀疏性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。