事件相机的特征可作运动线索,提升动态估计精度。
Static in Frames, Dynamic in Events: Rethinking Features in Event Cameras as Motion Cues

- 将结构张量特征与时空密度值结合,捕捉运动信息。
- 合成数据实验验证其对纹理和噪声鲁棒,且在数据少时增益显著。
- 集成至光流网络后,在DSEC真实场景中持续提效,尤其适合小模型。
事件相机以高时间分辨率异步捕捉亮度变化,需新型预处理方法应对下游任务。与静态图像快照不同,事件数据天然蕴含场景动态与物体运动信息,其特征行为在帧基视觉中无直接对应。本文分析事件角点检测中两种特征——结构张量的特征值与时空密度值——发现它们本质为运动线索。假设这些特征与局部几何信息结合,可增强运动估计。理论分析表明,运动角点处结构张量特征值与运动方向相关;合成数据上的受控实验验证:融合特征值与密度值能提供互补运动信息,且对纹理和拍摄噪声具有鲁棒性。最后,将所提特征集成至先进事件光流网络,在真实世界DSEC基准上评估,结果一致提升准确率,尤其在数据稀缺场景及低容量模型中表现更优。
原文摘要 · Abstract (English)
Event cameras capture intensity changes asynchronously with high temporal resolution, requiring novel preprocessing methods for downstream tasks. Unlike static intensity snapshots, event data inherently encode information about scene dynamics and object motion, meaning that features derived from events can exhibit behaviors with no direct analogue in frame-based vision. In this paper, we analyze two features used in event-based corner detection---the eigenvalues of the structure tensor and the spatiotemporal density values---and show that they are \emph{motion cues}. We hypothesize that these features, combined with local geometric information, can enhance motion estimation tasks. To validate this, we first theoretically analyze how the eigenvalues of the structure tensor at moving corner points relate to the direction of motion. We then design controlled experiments on a synthetic dataset, confirming that extending local geometric features with eigenvalues and density values provides complementary motion information and is robust to texture and shot noise. Finally, we integrate the proposed features into a state-of-the-art event-based optical flow network and evaluate on the real-world DSEC benchmark, where the added features consistently improve accuracy, with the largest gains in data-scarce scenarios and for lower-capacity models. The code for this paper can be found at: \href{https://github.com/hesamaraghi/static-in-frames-dynamic-in-events}{https://github.com/hesamaraghi/static-in-frames-dynamic-in-events}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。