用真实数据训练前端检测，合成数据优化3D重建，实现高鲁棒性乒乓球轨迹与旋转估计。
Uplifting Table Tennis: A Robust, Real-World Application for 3D Trajectory and Spin Estimation
- 分两阶段：前端用真实2D数据训练检测器，后端用物理正确合成数据训练3D提升模型
- 在真实视频上实现毫米级精度的3D轨迹和旋转估计，对缺失检测和帧率变化有强鲁棒性
- 适合体育分析、智能裁判等实际场景，尤其适用于存在噪声的单目视频
从标准单目视频中精确估计乒乓球的3D运动是一项挑战，因现有基于合成数据训练的方法难以泛化到真实世界中存在噪声、不完整的目标检测。主要原因在于真实视频缺乏3D轨迹和旋转标注。为此，我们提出一种新型两阶段流程，将问题分解为前端感知任务与后端2D到3D提升任务。该分离设计使前端组件可利用新构建的TTHQ数据集中的丰富2D监督信号进行训练，而后端提升网络仅在物理正确的合成数据上训练。我们特别重构了提升模型以抵御常见真实世界干扰，如检测缺失和帧率波动。通过集成球体检测器与台面关键点检测器，本方法将一个概念验证性的提升算法转化为实用、鲁棒且高性能的端到端3D乒乓球轨迹与旋转分析系统。
原文摘要 · Abstract (English)
Obtaining the precise 3D motion of a table tennis ball from standard monocular videos is a challenging problem, as existing methods trained on synthetic data struggle to generalize to the noisy, imperfect ball and table detections of the real world. This is primarily due to the inherent lack of 3D ground truth trajectories and spin annotations for real-world video. To overcome this, we propose a novel two-stage pipeline that divides the problem into a front-end perception task and a back-end 2D-to-3D uplifting task. This separation allows us to train the front-end components with abundant 2D supervision from our newly created TTHQ dataset, while the back-end uplifting network is trained exclusively on physically-correct synthetic data. We specifically re-engineer the uplifting model to be robust to common real-world artifacts, such as missing detections and varying frame rates. By integrating a ball detector and a table keypoint detector, our approach transforms a proof-of-concept uplifting method into a practical, robust, and high-performing end-to-end application for 3D table tennis trajectory and spin analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。