arXiv:2410.06698cs.CVcs.ET2024-10被引 8

用傅里叶变换识别动物行为,参数少五数量级却效果不俗

Fourier-based Action Recognition for Wildlife Behavior Quantification with Event Cameras

  • 基于傅里叶变换提取事件相机中的周期性运动模式
  • 在企鹅求偶行为数据集上达到接近深度网络的识别精度
  • 适合低资源设备部署,尤其适用于复杂自然场景

事件相机是一种仿生视觉传感器,以异步方式测量像素亮度变化而非固定帧率成像,具有高动态范围、低延迟和极小运动模糊的优点。现代计算机视觉算法多依赖神经网络,需将事件数据转为图像形式,难以充分发挥事件数据特性。本文提出基于傅里叶变换的动作识别方法,旨在捕捉自然界常见的周期性运动模式。特别应用于近期标注了‘兴奋展示’行为(企鹅特定频率扑翼)的企鹅数据集。结果表明,该方法简单有效,在未受控、多样化的数据下表现良好,识别性能略低于深度神经网络(DNN),但参数量仅为DNN的十万分之一(五数量级减少)。本工作为事件数据处理与动作识别提供了新思路。

原文摘要 · Abstract (English)

Event cameras are novel bio-inspired vision sensors that measure pixel-wise brightness changes asynchronously instead of images at a given frame rate. They offer promising advantages, namely a high dynamic range, low latency, and minimal motion blur. Modern computer vision algorithms often rely on artificial neural network approaches, which require image-like representations of the data and cannot fully exploit the characteristics of event data. We propose approaches to action recognition based on the Fourier Transform. The approaches are intended to recognize oscillating motion patterns commonly present in nature. In particular, we apply our approaches to a recent dataset of breeding penguins annotated for "ecstatic display", a behavior where the observed penguins flap their wings at a certain frequency. We find that our approaches are both simple and effective, producing slightly lower results than a deep neural network (DNN) while relying just on a tiny fraction of the parameters compared to the DNN (five orders of magnitude fewer parameters). They work well despite the uncontrolled, diverse data present in the dataset. We hope this work opens a new perspective on event-based processing and action recognition.

事件相机傅里叶变换行为识别低功耗

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。