构建驾驶模拟中的教学数据集，记录语言与动作协同学习过程。
SimCoachCorpus: A naturalistic dataset with language and trajectories for embodied teaching
- 采集29人90分钟驾驶数据，分有无教练指导组
- 含超2万条实时反馈、400+终端反馈和40小时交互数据
- 适合研究教学互动、运动学习建模与人机协作系统
高质量标注数据集对训练和评估人工智能方法至关重要，但在语言与身体动作交织的具身交互领域常显不足。尤其缺乏关于人们通过言语指导逐步掌握运动技能的数据。为此，我们提出SimCoachCorpus：一个基于赛车模拟器的自然情境数据集，支持对引导与非引导运动技能习得过程中丰富现象的研究。29名参与者在模拟赛道上驾驶约90分钟，其中15人接受专业性能教练的一对一指导，另14人无指导。数据集包含车辆状态与输入、赛道边界与理想路线、锥桶地标等信息，并同步教练的实时口头反馈及每圈结束后的终端反馈。我们还提供了每条实时反馈的高层次教学类别标注、学生遵从度评分，以及参与者自评的认知负荷与情绪状态（来自实验期间调查）。最终数据集包含超过20,000条实时反馈、400多条终端反馈和超过40小时的交互驾驶数据。该自然主义互动数据集可用于探究运动学习动态、分析语言现象，以及训练教学与学习的计算模型。我们展示了其在上下文学习、模仿学习与主题建模中的应用。数据托管于https://doi.org/10.7910/DVN/W7VTKZ，代码开源于https://github.com/ToyotaResearchInstitute/sim_coach_corpus。
原文摘要 · Abstract (English)
High-quality curated datasets are essential for training and evaluating AI approaches, but are often lacking in embodied interactive domains where language and physical action are intertwined. In particular, few datasets capture how people acquire motor skills in embodied tasks through verbal instruction over time. To address this gap, we introduce SimCoachCorpus: a unique dataset of race car simulator driving that enables the investigation of rich phenomena during guided and unguided motor skill acquisition. In this dataset, 29 humans were asked to drive in a driving simulator around a race track for approximately ninety minutes. Fifteen participants received one-on-one instruction from a professional performance driving coach, and 14 participants drove without coaching instruction. SimCoachCorpus includes features such as vehicle state and inputs, map (track boundaries and race-line), and cone landmarks. Additionally, these are synchronized with the coach's concurrent verbal feedback and additional terminal feedback at the end of each lap. We also provide high-quality annotations of high-level coaching categories for each concurrent feedback utterance, ratings on students' compliance with coaching advice, and self-reported cognitive load and emotional state of participants (gathered from surveys during the study). The final dataset includes over 20,000 concurrent feedback utterances, over 400 terminal feedback utterances, and over 40 hours of interactive driving data. Our naturalistic interactive dataset can be used to investigate motor learning dynamics, explore linguistic phenomena, and train computational models of teaching and learning. We demonstrate applications of this dataset for in-context learning, imitation learning, and topic modeling. Data is hosted at https://doi.org/10.7910/DVN/W7VTKZ and code is available at https://github.com/ToyotaResearchInstitute/sim_coach_corpus
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。