用深度学习生成逼真头动数据,解决元宇宙训练数据不足问题
Generating Realistic Synthetic Head Rotation Data for Extended Reality using Deep Learning
- 基于TimeGAN构建头旋转时序生成模型
- 生成数据与真实测量数据分布高度一致
- 适合元宇宙、虚拟现实系统训练与评估
扩展现实(Extended Reality)通过精准实时反映用户真实运动带来沉浸感和互动性。其中,头部旋转是关键运动信号,直接影响内容生成与传输。为实现无缝体验,需提前准确预测用户即将发生的头部旋转。训练与评估此类预测模型需要大量方位数据,而真人采集成本高昂。本文提出一种基于TimeGAN的头部旋转时序生成方法,该方法是生成对抗网络(GAN)在时序数据上的扩展,能够以少量真实数据为基础,生成大量符合原始数据分布的合成旋转序列,有效扩充数据集。
原文摘要 · Abstract (English)
Extended Reality is a revolutionary method of delivering multimedia content to users. A large contributor to its popularity is the sense of immersion and interactivity enabled by having real-world motion reflected in the virtual experience accurately and immediately. This user motion, mainly caused by head rotations, induces several technical challenges. For instance, which content is generated and transmitted depends heavily on where the user is looking. Seamless systems, taking user motion into account proactively, will therefore require accurate predictions of upcoming rotations. Training and evaluating such predictors requires vast amounts of orientational input data, which is expensive to gather, as it requires human test subjects. A more feasible approach is to gather a modest dataset through test subjects, and then extend it to a more sizeable set using synthetic data generation methods. In this work, we present a head rotation time series generator based on TimeGAN, an extension of the well-known Generative Adversarial Network, designed specifically for generating time series. This approach is able to extend a dataset of head rotations with new samples closely matching the distribution of the measured time series.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。