arXiv:2506.07860cs.CV2025-06CVPR被引 4

用事件相机+眼动数据实时预测乒乓球轨迹,延迟低至4.5毫秒。

Egocentric Event-Based Vision for Ping Pong Ball Trajectory Prediction

  • 基于眼动聚焦的事件视觉,只处理视网膜中心区域数据。
  • 检测延迟仅4.5毫秒,比传统30帧系统快14倍以上。
  • 首次实现从第一视角用事件相机预测乒乓球三维轨迹。

本文提出一种基于事件相机的实时第一人称乒乓球轨迹预测系统。与传统相机在高速运动下易产生运动模糊和高延迟不同,事件相机具备更高时间分辨率,可在对手击球后极短时间内完成状态更新,提升轨迹预测精度。研究团队采集了包含球体三维真实轨迹的数据集,同步记录了Meta Project Aria眼镜的眼动数据与事件流。系统采用类生物视锥聚焦机制,利用眼镜获取的眼动信息,仅处理视野中心区域的事件数据,显著降低计算负载,在所收集轨迹上实现10.81倍的延迟下降。整个检测流水线最坏情况总延迟仅为4.5毫秒,远低于传统30帧系统仅感知环节就达66毫秒的水平。最后,通过拟合轨迹预测模型,实现未来三维轨迹的生成。据我们所知,这是首个从第一人称视角使用事件相机进行乒乓球轨迹预测的工作。

原文摘要 · Abstract (English)

In this paper, we present a real-time egocentric trajectory prediction system for table tennis using event cameras. Unlike standard cameras, which suffer from high latency and motion blur at fast ball speeds, event cameras provide higher temporal resolution, allowing more frequent state updates, greater robustness to outliers, and accurate trajectory predictions using just a short time window after the opponent's impact. We collect a dataset of ping-pong game sequences, including 3D ground-truth trajectories of the ball, synchronized with sensor data from the Meta Project Aria glasses and event streams. Our system leverages foveated vision, using eye-gaze data from the glasses to process only events in the viewer's fovea. This biologically inspired approach improves ball detection performance and significantly reduces computational latency, as it efficiently allocates resources to the most perceptually relevant regions, achieving a reduction factor of 10.81 on the collected trajectories. Our detection pipeline has a worst-case total latency of 4.5 ms, including computation and perception - significantly lower than a frame-based 30 FPS system, which, in the worst case, takes 66 ms solely for perception. Finally, we fit a trajectory prediction model to the estimated states of the ball, enabling 3D trajectory forecasting in the future. To the best of our knowledge, this is the first approach to predict table tennis trajectories from an egocentric perspective using event cameras.

事件相机轨迹预测眼动追踪实时系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。