arXiv:2606.19534cs.CVcs.AI2026-06被引 1

用扩散模型实现多区域并行感知,速度更快且效果不降。

PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models

论文配图:PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models
图 1 · 摘自论文原文
  • 采用扩散语言模型并行生成多个区域描述,突破传统逐个处理的效率瓶颈。
  • 在多区域图像描述任务中,推理速度提升显著,同时保持高质量输出。
  • 适合需要高效多区域视觉理解的应用,如智能监控与医学影像分析。

多模态大语言模型(MLLM)在视觉理解任务中取得显著进展,但多数依赖自回归生成,限制了其在需对多个区域进行描述的任务中的效率。本文提出PerceptionDLM,一种专为高效并行区域感知优化的多模态扩散语言模型。基于PerceptionDLM-Base这一开源扩散型MLLM基线,其性能已达开源模型前列。通过引入高效的提示设计与结构化注意力掩码,模型可同时感知多个被遮蔽区域,在序列和词元层面实现并行生成。该设计显著提升推理效率,优于传统顺序处理方法。为系统评估扩散语言模型的并行感知能力,我们构建了新的平行详细局部描述基准(ParaDLC-Bench),将DLC-Bench扩展为每图含多个区域掩码,支持对生成质量与推理效率的联合评估。实验表明,PerceptionDLM在区域描述任务中保持竞争力,同时在多区域感知任务中实现显著提速。结果表明,多模态扩散语言模型在高效并行视觉感知方面具有巨大潜力。据我们所知,这是首个利用扩散语言模型实现并行区域描述与感知的工作。代码、模型与数据集均已公开。

原文摘要 · Abstract (English)

Multimodal large language models (MLLMs) have achieved remarkable progress in visual understanding tasks. However, most existing MLLMs rely on autoregressive generation, which limits their efficiency for perception tasks that require captioning multiple regions. In this work, we propose PerceptionDLM, a multimodal diffusion language model optimized for efficient parallel region perception. Built upon PerceptionDLM-Base, a strong foundational baseline that achieves state-of-the-art performance among open-source diffusion MLLMs, our architecture fully leverages the parallel decoding nature of DLMs. Specifically, we introduce efficient prompting and structured attention masking to enable simultaneous perception of multiple masked regions, allowing the model to generate region descriptions in parallel at both the sequence and token levels. This design significantly improves inference efficiency compared with existing approaches that process regions sequentially. To systematically evaluate the parallelism property of visual perception capability for DLMs, we construct a new Parallel Detailed Localized Captioning Benchmark (ParaDLC-Bench) by scaling the DLC-Bench to include multiple region masks per image, enabling joint evaluation of both caption quality and inference efficiency. Experiments demonstrate that PerceptionDLM maintains competitive performance in region captioning while achieving substantial speed improvements for multi-region perception tasks. Our results highlight the potential of multimodal diffusion language models for efficient, parallel visual perception. To the best of our knowledge, we are the first to achieve parallel region caption and perception by leveraging the advantages of diffusion language models. Code, models, and datasets are released.

多模态扩散模型并行感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。