对比三种AI队友熟悉方法,发现结合文档与实操最有效。
Model Cards for AI Teammates: Comparing Human-AI Team Familiarization Methods for High-Stakes Environments
- 用文档、训练或无熟悉三组对比,测试人机协作效率。
- 文档组策略形成最快但过于保守,交互组更敢冒险且策略灵活。
- 建议融合文档、在岗培训与探索互动,适配不同风险偏好用户。
我们在一项包含60名参与者的事先-事后对照实验中,比较了三种在高压力情报监视侦察(ISR)环境中熟悉人工智能(AI)队友的方法:阅读文档、与AI共同训练、或不进行任何熟悉。结果表明,关于AI决策算法及其相对于人类优劣势的信息最为关键,能帮助团队快速制定复杂协作策略。仅读文档的组别策略采纳速度最快,但表现出过度保守行为,影响得分;通过实际互动熟悉者虽对内部机制理解较弱,但能通过观察推断相似信息,更愿意尝试不同控制模式并承担风险。个体风险偏好与交互方式间存在显著差异,提示人机控制界面设计需考虑个性差异。研究建议采用结合文档说明、结构化现场训练和探索性交互的综合熟悉方法。
原文摘要 · Abstract (English)
We compare three methods of familiarizing a human with an artificial intelligence (AI) teammate ("agent") prior to operation in a collaborative, fast-paced intelligence, surveillance, and reconnaissance (ISR) environment. In a between-subjects user study (n=60), participants either read documentation about the agent, trained alongside the agent prior to the mission, or were given no familiarization. Results showed that the most valuable information about the agent included details of its decision-making algorithms and its relative strengths and weaknesses compared to the human. This information allowed the familiarization groups to form sophisticated team strategies more quickly than the control group. Documentation-based familiarization led to the fastest adoption of these strategies, but also biased participants towards risk-averse behavior that prevented high scores. Participants familiarized through direct interaction were able to infer much of the same information through observation, and were more willing to take risks and experiment with different control modes, but reported weaker understanding of the agent's internal processes. Significant differences were seen between individual participants' risk tolerance and methods of AI interaction, which should be considered when designing human-AI control interfaces. Based on our findings, we recommend a human-AI team familiarization method that combines AI documentation, structured in-situ training, and exploratory interaction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。