用视频分析预测虚拟现实晕动症严重程度,准确率达68.4%。
Towards Cybersickness Severity Classification from VR Gameplay Videos Using Transfer Learning and Temporal Modeling
- 用InceptionV3提取视频视觉特征,再用LSTM捕捉时间动态。
- 在真实场景中实现68.4%的晕动症分级准确率。
- 适合VR开发者优化用户体验,推动视频驱动的舒适性研究。
随着虚拟现实技术快速发展,其在医疗、教育和娱乐等领域的应用日益广泛。然而,与运动病相似的晕动症症状持续影响用户接受度。尽管已有研究利用眼动、头动等传感器数据进行多模态深度学习建模,但基于视频特征预测晕动症的研究仍较少。本文提出一种新方法:使用预训练于ImageNet的InceptionV3模型提取VR游戏视频的高层视觉特征,并输入长短期记忆网络(LSTM)以捕捉虚拟体验的时间动态,进而预测晕动症严重程度。该方法充分利用视频的时间序列特性,在实验中达到68.4%的分类准确率,优于仅依赖视频数据的现有模型。结果为VR开发者提供了一种实用工具,用于评估和缓解虚拟环境中的晕动症问题,并为未来基于视频的时序建模研究奠定基础。
原文摘要 · Abstract (English)
With the rapid advancement of virtual reality (VR) technology, its adoption across domains such as healthcare, education, and entertainment has grown significantly. However, the persistent issue of cybersickness, marked by symptoms resembling motion sickness, continues to hinder widespread acceptance of VR. While recent research has explored multimodal deep learning approaches leveraging data from integrated VR sensors like eye and head tracking, there remains limited investigation into the use of video-based features for predicting cybersickness. In this study, we address this gap by utilizing transfer learning to extract high-level visual features from VR gameplay videos using the InceptionV3 model pretrained on the ImageNet dataset. These features are then passed to a Long Short-Term Memory (LSTM) network to capture the temporal dynamics of the VR experience and predict cybersickness severity over time. Our approach effectively leverages the time-series nature of video data, achieving a 68.4% classification accuracy for cybersickness severity. This surpasses the performance of existing models trained solely on video data, providing a practical tool for VR developers to evaluate and mitigate cybersickness in virtual environments. Furthermore, this work lays the foundation for future research on video-based temporal modeling for enhancing user comfort in VR applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。