arXiv:2602.03230cs.CV2026-02被引 4

提出高效事件视觉大模型,显著提升处理速度并支持超长序列。

EventFlash: Towards Efficient MLLMs for Event-Based Vision

  • 通过自适应时间窗口压缩与稀疏注意力机制,减少事件流冗余数据。
  • 推理速度提升12.4倍,支持最长1000个时间桶的序列处理。
  • 适合需要高速低光感知的自动驾驶、机器人等场景使用。

事件视觉多模态大模型(MLLM)在高速和低光环境下具备鲁棒感知能力,克服了传统帧基MLLM的局限。然而现有方法仍依赖密集图像式处理,忽视事件流的时空稀疏性,导致计算开销高。本文提出EventFlash,通过时空标记稀疏化实现高效处理。技术上,构建包含超过50万条指令的EventMind大规模多样场景数据集,支持课程训练;设计自适应时间窗口聚合模块,动态压缩时间标记并保留关键时序信息;引入稀疏密度引导注意力模块,聚焦有效空间区域,抑制空旷或稀疏区域。实验表明,EventFlash相比基线(EventFlash-Zero)实现12.4倍吞吐量提升,支持长达1,000个时间桶的序列处理,远超EventGPT的5个时间桶限制。我们相信EventFlash可作为事件视觉的高效基础模型。

原文摘要 · Abstract (English)

Event-based multimodal large language models (MLLMs) enable robust perception in high-speed and low-light scenarios, addressing key limitations of frame-based MLLMs. However, current event-based MLLMs often rely on dense image-like processing paradigms, overlooking the spatiotemporal sparsity of event streams and resulting in high computational cost. In this paper, we propose EventFlash, a novel and efficient MLLM to explore spatiotemporal token sparsification for reducing data redundancy and accelerating inference. Technically, we build EventMind, a large-scale and scene-diverse dataset with over 500k instruction sets, providing both short and long event stream sequences to support our curriculum training strategy. We then present an adaptive temporal window aggregation module for efficient temporal sampling, which adaptively compresses temporal tokens while retaining key temporal cues. Finally, a sparse density-guided attention module is designed to improve spatial token efficiency by selecting informative regions and suppressing empty or sparse areas. Experimental results show that EventFlash achieves a $12.4\times$ throughput improvement over the baseline (EventFlash-Zero) while maintaining comparable performance. It supports long-range event stream processing with up to 1,000 bins, significantly outperforming the 5-bin limit of EventGPT. We believe EventFlash serves as an efficient foundation model for event-based vision.

事件视觉大模型稀疏化高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。