arXiv:2603.10680cs.HCcs.AI2026-03被引 1

构建可跨平台的多模态数字人模型框架,支持无障碍交互研究。

A Platform-Agnostic Multimodal Digital Human Modelling Framework: Neurophysiological Sensing in Game-Based Interaction

  • 分离感知、交互建模与推理,实现跨平台复现
  • 整合脑电肌电等多源生理信号,同步采集五类数据流
  • 适用于无障碍设计与自适应系统研究,无需修改架构

数字人建模(DHM)正受到人工智能、可穿戴生物传感和交互式数字环境的推动,尤其在提升可及性与包容性方面。然而,许多基于AI的DHM方法仍高度依赖特定平台、任务或分析流程,限制了可复现性、可扩展性与伦理再利用。本文提出一种平台无关的DHM框架,通过显式分离感知、交互建模与推理就绪性,支持面向AI的多模态交互研究。该框架采用OpenBCI Galea头戴设备作为统一多模态感知层,同步采集EEG、EMG、EOG、PPG与惯性数据流,并基于SuperTux构建可复现的游戏化交互环境。不嵌入AI模型或行为推断,仅将生理信号表示为结构化、时间对齐的可观测变量,使下游AI方法可在合规伦理审批下应用。交互通过计算任务原语与时间戳事件标记建模,确保异构传感器与平台间的对齐一致性。作者自测验证了数据完整性、流连续性与同步性;未报告人类受试者评估或AI推断结果。讨论了数据吞吐量、延迟及扩展至更多传感器或交互模态的可扩展性。示例用例展示该框架支持无障碍交互设计与自适应系统研究,且无需架构修改。该框架为未来伦理批准的包容性DHM研究提供新兴技术基础设施。

原文摘要 · Abstract (English)

Digital Human Modelling (DHM) is increasingly shaped by advances in AI, wearable biosensing, and interactive digital environments, particularly in research addressing accessibility and inclusion. However, many AI-enabled DHM approaches remain tightly coupled to specific platforms, tasks, or interpretative pipelines, limiting reproducibility, scalability, and ethical reuse. This paper presents a platform-agnostic DHM framework designed to support AI-ready multimodal interaction research by explicitly separating sensing, interaction modelling, and inference readiness. The framework integrates the OpenBCI Galea headset as a unified multimodal sensing layer, providing concurrent EEG, EMG, EOG, PPG, and inertial data streams, alongside a reproducible, game-based interaction environment implemented using SuperTux. Rather than embedding AI models or behavioural inference, physiological signals are represented as structured, temporally aligned observables, enabling downstream AI methods to be applied under appropriate ethical approval. Interaction is modelled using computational task primitives and timestamped event markers, supporting consistent alignment across heterogeneous sensors and platforms. Technical verification via author self-instrumentation confirms data integrity, stream continuity, and synchronisation; no human-subjects evaluation or AI inference is reported. Scalability considerations are discussed with respect to data throughput, latency, and extension to additional sensors or interaction modalities. Illustrative use cases demonstrate how the framework can support AI-enabled DHM and HCI studies, including accessibility-oriented interaction design and adaptive systems research, without requiring architectural modifications. The proposed framework provides an emerging-technology-focused infrastructure for future ethics-approved, inclusive DHM research.

数字人建模多模态感知可穿戴设备无障碍交互

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。