用状态空间模型高效重建事件流图像，速度更快、内存更省。
EmambaIR: Efficient Visual State Space Model for Event-guided Image Reconstruction
- 引入稀疏注意力与门控状态空间模块，实现低复杂度跨模态融合。
- 在六大数据集上超越现有方法，计算量降低至O(n)级别。
- 适合高分辨率图像重建任务，尤其适用于实时系统部署。
基于事件的图像重建方法通常依赖卷积神经网络（CNN）和视觉变换器（ViT）处理互补事件信息。然而，这类架构存在根本性局限：CNN难以捕捉全局特征相关性，而ViT的计算复杂度为二次方级（如O(n²)），限制了其在高分辨率场景中的应用。为此，我们提出EmambaIR，一种面向时空稀疏且连续事件流的高效视觉状态空间模型。框架包含两个关键组件：跨模态Top-k稀疏注意力模块（TSAM）和门控状态空间模块（GSSM）。TSAM通过像素级Top-k稀疏注意力引导跨模态交互，生成丰富而稀疏的融合特征；GSSM利用非线性门控单元增强原始线性复杂度（O(n)）状态空间模型的时序表征能力，有效捕获全局上下文依赖，且无典型计算开销。在六个数据集上的三类重建任务（运动去模糊、去雨、高动态范围增强）的实验表明，EmambaIR显著优于当前最优方法，同时大幅降低内存占用与计算成本。代码与数据已公开于：https://github.com/YunhangWickert/EmambaIR
原文摘要 · Abstract (English)
Recent event-based image reconstruction methods predominantly rely on Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) to process complementary event information. However, these architectures face fundamental limitations: CNNs often fail to capture global feature correlations, whereas ViTs incur quadratic computational complexity (e.g., $O(n^2)$), hindering their application in high-resolution scenarios. To address these bottlenecks, we introduce EmambaIR, an Efficient visual State Space Model designed for image reconstruction using spatially sparse and temporally continuous event streams. Our framework introduces two key components: the cross-modal Top-k Sparse Attention Module (TSAM) and the Gated State-Space Module (GSSM). TSAM efficiently performs pixel-level top-k sparse attention to guide cross-modal interactions, yielding rich yet sparse fusion features. Subsequently, GSSM utilizes a nonlinear gated unit to enhance the temporal representation of vanilla linear-complexity ($O(n)$) SSMs, effectively capturing global contextual dependencies without the typical computational overhead. Extensive experiments on six datasets across three diverse image reconstruction tasks - motion deblurring, deraining, and High Dynamic Range (HDR) enhancement - demonstrate that EmambaIR significantly outperforms state-of-the-art methods while offering substantial reductions in memory consumption and computational cost. The source code and data are publicly available at: https://github.com/YunhangWickert/EmambaIR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。