用状态空间模型让事件相机数据复用现有图像模型，提升性能。
Event USKT : U-State Space Model in Knowledge Transfer for Event Cameras
- 设计U形状态空间框架，将事件数据转为可适配RGB模型的输入
- 在三个数据集上分别提升0.95%~3.57%，参数量小且效果显著
- 适合想用少量事件数据快速部署模型的研究者或工程师
事件相机作为新兴成像技术，相比传统RGB相机具有更低能耗和更高帧率的优势。然而，可用事件数据量有限，制约了其发展。为此，我们提出专用于事件到RGB知识迁移的U形状态空间模型（USKT）框架。该框架生成兼容RGB帧的输入，使事件数据能有效复用预训练的RGB模型，并在极少调参下实现竞争力表现。框架内提出的双向逆状态空间模型（BiR-SSM）采用共享权重策略，相比传统双向扫描机制更高效，节省计算资源。在DVS128 Gesture、N-Caltech101和CIFAR-10-DVS数据集上，结合ResNet50作为主干网络时，性能分别提升0.95%、3.57%和2.9%，验证了USKT的适应性与有效性。代码将在论文接受后公开。
原文摘要 · Abstract (English)
Event cameras, as an emerging imaging technology, offer distinct advantages over traditional RGB cameras, including reduced energy consumption and higher frame rates. However, the limited quantity of available event data presents a significant challenge, hindering their broader development. To alleviate this issue, we introduce a tailored U-shaped State Space Model Knowledge Transfer (USKT) framework for Event-to-RGB knowledge transfer. This framework generates inputs compatible with RGB frames, enabling event data to effectively reuse pre-trained RGB models and achieve competitive performance with minimal parameter tuning. Within the USKT architecture, we also propose a bidirectional reverse state space model. Unlike conventional bidirectional scanning mechanisms, the proposed Bidirectional Reverse State Space Model (BiR-SSM) leverages a shared weight strategy, which facilitates efficient modeling while conserving computational resources. In terms of effectiveness, integrating USKT with ResNet50 as the backbone improves model performance by 0.95%, 3.57%, and 2.9% on DVS128 Gesture, N-Caltech101, and CIFAR-10-DVS datasets, respectively, underscoring USKT's adaptability and effectiveness. The code will be made available upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。