arXiv:2605.20680cs.CV2026-05

事件相机+陀螺仪,解决暗光抖动下的动作识别难题

DarkShake-DVS: Event-based Human Action Recognition under Low-light andShaking Camera Conditions

论文配图:DarkShake-DVS: Event-based Human Action Recognition under Low-light andShaking Camera Conditions
图 1 · 摘自论文原文
  • 用非线性扭曲补偿相机抖动,重建清晰输入帧
  • 在18,041段真实场景视频上实现更优识别准确率
  • 专为低光强晃动环境设计,适合智能监控等场景

人类动作识别是计算机视觉基础任务,广泛应用于实际场景。现实部署常面临低光照与自由6-DoF相机运动,导致图像质量下降、时间连贯性破坏,现有方法可靠性受限。事件相机具备高灵敏度与微秒级时间分辨率,结合惯性测量单元(IMU)提供潜在解决方案。然而当前研究存在两大挑战:缺乏集成低光、6-DoF运动及同步IMU数据的基准;缺乏有效运动补偿技术。为此,我们提出事件-IMU稳定化动作识别(EIS-HAR),包含两个模块:其一为EIS模块,通过非线性扭曲函数减少运动模糊,重建运动补偿输入;其二为四阶段混合架构的HAR模块,高效提取时空特征以实现精准识别。为缓解数据稀缺问题,引入DarkShake-DVS——首个大规模事件相机动作识别基准,包含18,041段真实世界片段,覆盖低光与强烈6-DoF运动,并附带同步IMU数据。三组数据集上的实验表明,EIS-HAR持续优于当前最优方法。

原文摘要 · Abstract (English)

Human Action Recognition (HAR) is a fundamental computer vision task with diverse real-world applications. Practical deployments often involve low-light environments and unconstrained 6-DoF camera motion, conditions that degrade visual quality, disrupt temporal coherence, and compromise reliability of existing methods. Event cameras, with high low-light sensitivity and microsecond-level temporal resolution, paired with an inertial measurement unit (IMU), present a promising solution. However, current research faces two key challenges: absence of a benchmark integrating low-light conditions, 6-DoF motion, and synchronized IMU data; and lack of effective motion compensation techniques. To address these, we propose Event-IMU Stabilized HAR (EIS-HAR), with two modules. The first is an EIS module that reduces motion blur via a non-linear warping function to reconstruct a motion-compensated input. The second is a HAR module with a four-stage hybrid architecture to efficiently extract spatiotemporal features for accurate action recognition. To alleviate data scarcity, we introduce DarkShake-DVS, the first large-scale event-based HAR benchmark that includes 18,041 realworld clips captured in low light and intense 6-DoF motion, supplemented by synchronized IMU data. Extensive experiments on three datasets demonstrate consistent superiority of EIS-HAR over state-of-the-art methods.

动作识别事件相机低光环境运动补偿

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。