轻量级事件数据显著性预测模型，压缩超500倍仍保持高精度。
SED:Lightweight Saliency prediction for Event-based data via Distillation
- 基于深度可分离时空卷积块，设计轻量网络结构。
- 模型大小缩小562倍，参数减少554倍，性能持平或超越基准。
- 知识蒸馏提升泛化能力，适用于真实事件数据场景。
事件数据显著性预测近年受到关注，结合事件相机与显著性估计可作为边缘计算中下游感知的高效预处理阶段。然而现有方法或为类脑架构、在事件显著性基准上表现不佳，或因依赖变换器或3D卷积而过于臃肿，难以部署于资源受限的边缘设备。受高效卷积模块启发，本文提出名为SED的轻量级网络，通过知识蒸馏训练，核心为深度可分离时空卷积块（DSTconv），该模块对3D深度可分离卷积进行分解。相比教师模型，其模型大小从180 MB降至0.32 MB（压缩562倍），参数量从45M降至81k（压缩554倍），同时在N-DHF1K和N-UCF Sports数据集上达到或超过教师性能。此外，模型在分布外泛化能力强，能从合成事件数据迁移到真实事件数据，而端到端训练的模型则失败。
原文摘要 · Abstract (English)
Event-based saliency prediction has gained attention recently, as combining event cameras with saliency estimation can act as an upstream stage that naturally improves the efficiency of downstream eventbased perception at the edge. However, current approaches are either neuromorphic, underperforming on event-based saliency benchmarks, or too heavy for resource-constrained edge applications due to their reliance on transformers or 3D convolutions. Drawing inspiration from efficient convolutional modules, SED and aiming to exploit the temporal information in event data, we propose a lightweight network, trained through knowledge distillation, built on a Depthwise Spatio-Temporal Block (DSTconv) -- a factorization of the 3D depthwise separable convolution. Relative to its teacher, our model reduces the model size from 180 MB to 0.32 MB (562x) and the parameter count from 45M to 81k (554x), while matching or outperforming it on the N-DHF1K and N-UCF Sports datasets. Moreover, it generalizes strongly beyond its training distribution, transferring from synthetic to real event data where a model trained from scratch fails.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。