用大模型分析元宇宙用户行为,跨虚拟现实场景都适用
Explainable XR: Understanding User Behaviors of XR Environments using LLM-assisted Analytics Framework
- 设计通用行为记录格式UAD,捕捉动作、意图和上下文
- 支持跨AR/VR/MR的多用户协作数据采集与分析
- 结合大模型提供可解释的可视化洞察,适合产品与研究者
我们提出 Explainable XR,一个端到端框架,利用大语言模型(LLMs)辅助解析扩展现实(XR)环境中用户行为。现有分析框架难以应对跨虚拟现实场景(如AR、VR、MR)转换、多人协作应用及多模态数据复杂性等问题。Explainable XR通过三个核心组件解决:(1) 新型用户数据记录格式——用户行为描述符(User Action Descriptor, UAD),可捕获用户多模态动作、意图与上下文;(2) 平台无关的XR会话记录器;(3) 支持大模型辅助解读的可视化分析界面,按分析师视角生成可操作洞察。我们在五种使用场景中验证了该框架在个体与协作式应用中的普适性。技术评估与用户研究表明,Explainable XR能有效理解用户行为,提供多维度、可行动的洞察。
原文摘要 · Abstract (English)
We present Explainable XR, an end-to-end framework for analyzing user behavior in diverse eXtended Reality (XR) environments by leveraging Large Language Models (LLMs) for data interpretation assistance. Existing XR user analytics frameworks face challenges in handling cross-virtuality - AR, VR, MR - transitions, multi-user collaborative application scenarios, and the complexity of multimodal data. Explainable XR addresses these challenges by providing a virtuality-agnostic solution for the collection, analysis, and visualization of immersive sessions. We propose three main components in our framework: (1) A novel user data recording schema, called User Action Descriptor (UAD), that can capture the users' multimodal actions, along with their intents and the contexts; (2) a platform-agnostic XR session recorder, and (3) a visual analytics interface that offers LLM-assisted insights tailored to the analysts' perspectives, facilitating the exploration and analysis of the recorded XR session data. We demonstrate the versatility of Explainable XR by demonstrating five use-case scenarios, in both individual and collaborative XR applications across virtualities. Our technical evaluation and user studies show that Explainable XR provides a highly usable analytics solution for understanding user actions and delivering multifaceted, actionable insights into user behaviors in immersive environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。