生成可控制姿势的3D骑行者合成数据,解决自动驾驶中骑行者数据少的问题。
3DArticCyclists: Generating Synthetic Articulated 8D Pose-Controllable Cyclist Data for Computer Vision Applications
- 基于3D高斯溅射构建可参数化控制的8自由度骑行者模型
- 通过关键点优化实现人物与自行车的自然姿态融合
- 生成数据可用于骑行者姿态估计、意图预测等任务
在自动驾驶感知中,骑行者是关键安全对象。现有公开数据集虽含大量车辆但骑行者样本稀少,且外观与姿态多样性不足。这限制了深度学习模型在骑行者语义分割、姿态估计及过街意图预测中的泛化能力,也阻碍了细粒度姿态估计和人-刚体复杂交互的时空分析研究。为此,本文提出一个生成合成动态3D骑行者数据的框架:首先构建基于部件的多视角3D自行车数据集3DArticBikes,训练3D高斯溅射(3DGS)重建与渲染方法;接着设计参数化3DGS组合模型,实现8自由度姿态可控的3D自行车;最后结合骑行视频动态信息,通过3D关键点优化的逆运动学算法,将可选合成人物自动匹配到自行车上,生成完整动态3D骑行者。定性与定量结果表明,生成效果优于近期基于稳定扩散的方法。
原文摘要 · Abstract (English)
In Autonomous Driving (AD) Perception, cyclists are considered safety-critical scene objects. Commonly used publicly-available AD datasets typically contain large amounts of car and vehicle object instances but a low number of cyclist instances, usually with limited appearance and pose diversity. This cyclist training data scarcity problem not only limits the generalization of deep-learning perception models for cyclist semantic segmentation, pose estimation, and cyclist crossing intention prediction, but also limits research on new cyclist-related tasks such as fine-grained cyclist pose estimation and spatio-temporal analysis under complex interactions between humans and articulated objects. To address this data scarcity problem, in this paper we propose a framework to generate synthetic dynamic 3D cyclist data assets that can be used to generate training data for different tasks. In our framework, we designed a methodology for creating a new part-based multi-view articulated synthetic 3D bicycle dataset that we call 3DArticBikes that we use to train a 3D Gaussian Splatting (3DGS)-based reconstruction and image rendering method. We then propose a parametric bicycle 3DGS composition model to assemble 8-DoF pose-controllable 3D bicycles. Finally, using dynamic information from cyclist videos, we build a complete synthetic dynamic 3D cyclist (rider pedaling a bicycle) by re-posing a selectable synthetic 3D person, while automatically placing the rider onto one of our new articulated 3D bicycles using a proposed 3D Keypoint optimization-based Inverse Kinematics pose refinement. We present both, qualitative and quantitative results where we compare our generated cyclists against those from a recent stable diffusion-based method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。