arXiv:2505.02393cs.CV2025-05被引 3

用合成事件信息增强视频异常检测,无需专用传感器。

Uncertainty-Weighted Image-Event Multimodal Fusion for Video Anomaly Detection

论文配图:Uncertainty-Weighted Image-Event Multimodal Fusion for Video Anomaly Detection
图 1 · 摘自论文原文
  • 通过学生t分布建模噪声,用逆方差权重融合图像与事件特征
  • 在多个真实数据集上超越现有方法,最高提升6.2%准确率
  • 适合无事件传感器的视频分析场景,尤其关注瞬时运动异常

现有视频异常检测方法多依赖RGB帧,难以捕捉突发或短暂的运动线索,而这些是异常事件的关键指标。为此,我们提出图像-事件融合视频异常检测框架IEF-VAD,直接从RGB视频生成事件表示,并通过一种基于不确定性的原理化过程将其与图像特征融合。该系统(i)采用学生t分布建模重尾传感器噪声,通过拉普拉斯近似推导出逐值逆方差权重;(ii)采用卡尔曼式逐帧更新机制,在时间上平衡多模态信息;(iii)迭代优化融合后的潜在状态,消除残余跨模态噪声。在未使用任何专用事件传感器或帧级标签的情况下,IEF-VAD在多个真实世界异常检测基准上达到新最优性能。结果表明,合成事件表示能有效突出RGB帧中常被忽略的运动线索,实现无需专用传感器的精准、鲁棒视频理解。代码与模型已开源:https://github.com/EavnJeong/IEF-VAD。

原文摘要 · Abstract (English)

Most existing video anomaly detectors rely solely on RGB frames, which lack the temporal resolution needed to capture abrupt or transient motion cues, key indicators of anomalous events. To address this limitation, we propose Image-Event Fusion for Video Anomaly Detection (IEF-VAD), a framework that synthesizes event representations directly from RGB videos and fuses them with image features through a principled, uncertainty-aware process. The system (i) models heavy-tailed sensor noise with a Student`s-t likelihood, deriving value-level inverse-variance weights via a Laplace approximation; (ii) applies Kalman-style frame-wise updates to balance modalities over time; and (iii) iteratively refines the fused latent state to erase residual cross-modal noise. Without any dedicated event sensor or frame-level labels, IEF-VAD sets a new state of the art across multiple real-world anomaly detection benchmarks. These findings highlight the utility of synthetic event representations in emphasizing motion cues that are often underrepresented in RGB frames, enabling accurate and robust video understanding across diverse applications without requiring dedicated event sensors. Code and models are available at https://github.com/EavnJeong/IEF-VAD.

异常检测多模态融合事件相机视频理解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。