用虚拟模型训练3D牛体关节估计算法,实现真实牛群姿态分析。
Sim2real Cattle Joint Estimation in 3D point clouds
- 基于3D动画模型生成多样化牛体姿态,缩小仿真与现实差距。
- 通过表面测地距离与多边定位,精准提取内部关节位置。
- 可预测牛髋高,助力农业牲畜健康监测。
理解牛的健康状况在农业中至关重要。牛的体型和关节运动蕴含关键健康信息,但构建用于3D体姿估计的全面数据集面临巨大挑战。本研究专注于构建专为牛设计的3D点云数据集。借助数字艺术家的专业能力,使用单一动画3D模型模拟多种牛体姿态,并通过扩展模型形状以覆盖不同体态,缩小“仿真到现实”的差异。利用这些标注模型训练深度学习框架,仅依据外部表面曲率估计内部关节位置。方法上结合表面流形上的测地距离与多边定位,在语义关键点检测编码器-解码器架构中实现关节提取。通过对比真实牛群在挤入通道中行走时的关节连接长度,验证了该方法的鲁棒性。受哺乳动物骨骼长度与整体身高间的已知异速生长关系启发,我们利用估计的关节位置在真实牛数据集中预测髋高,拓展了该方法在提升牛只监控实践中的应用价值。
原文摘要 · Abstract (English)
Understanding the well-being of cattle is crucial in various agricultural contexts. Cattle's body shape and joint articulation carry significant information about their welfare, yet acquiring comprehensive datasets for 3D body pose estimation presents a formidable challenge. This study delves into the construction of such a dataset specifically tailored for cattle. Leveraging the expertise of digital artists, we use a single animated 3D model to represent diverse cattle postures. To address the disparity between virtual and real-world data, we augment the 3D model's shape to encompass a range of potential body appearances, thereby narrowing the "sim2real" gap. We use these annotated models to train a deep-learning framework capable of estimating internal joints solely based on external surface curvature. Our contribution is specifically the use of geodesic distance over the surface manifold, coupled with multilateration to extract joints in a semantic keypoint detection encoder-decoder architecture. We demonstrate the robustness of joint extraction by comparing the link lengths extracted on real cattle mobbing and walking within a race. Furthermore, inspired by the established allometric relationship between bone length and the overall height of mammals, we utilise the estimated joints to predict hip height within a real cattle dataset, extending the utility of our approach to offer insights into improving cattle monitoring practices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。