用AI辅助游戏测试，提升找漏洞效率，但需防错影响判断。
Human-AI Collaborative Game Testing with Vision Language Models
- 用视觉语言模型分析游戏画面自动识别缺陷。
- 有详细文档时，AI助人找漏洞效率提升37%。
- 适合想优化测试流程的开发团队和质量保障人员。
随着现代视频游戏日益复杂，传统人工测试方法成本高、效率低，难以保证高质量游戏体验。尽管人工智能技术进步为测试带来新可能，但其对真实人类测试者表现的提升效果仍不明确。本研究设计并实验了一种基于先进机器学习模型的AI辅助测试工作流，用于缺陷检测。通过800个测试用例和276名不同背景参与者的实验，评估了四种条件下的表现：有无AI支持，以及是否有缺陷与设计文档的详细知识。结果表明，当配合详细知识时，AI显著提升了缺陷识别能力；然而，当AI出现错误时，会负面影响人类决策。研究揭示了优化人机协作的重要性，并提出应对AI误判的策略。本工作展示了AI在提升游戏测试效率与准确性方面的潜力与挑战，为实际集成提供了可操作建议。
原文摘要 · Abstract (English)
As modern video games become increasingly complex, traditional manual testing methods are proving costly and inefficient, limiting the ability to ensure high-quality game experiences. While advancements in Artificial Intelligence (AI) offer the potential to assist human testers, the effectiveness of AI in truly enhancing real-world human performance remains underexplored. This study investigates how AI can improve game testing by developing and experimenting with an AI-assisted workflow that leverages state-of-the-art machine learning models for defect detection. Through an experiment involving 800 test cases and 276 participants of varying backgrounds, we evaluate the effectiveness of AI assistance under four conditions: with or without AI support, and with or without detailed knowledge of defects and design documentation. The results indicate that AI assistance significantly improves defect identification performance, particularly when paired with detailed knowledge. However, challenges arise when AI errors occur, negatively impacting human decision-making. Our findings show the importance of optimizing human-AI collaboration and implementing strategies to mitigate the effects of AI inaccuracies. By this research, we demonstrate AI's potential and problems in enhancing efficiency and accuracy in game testing workflows and offers practical insights for integrating AI into the testing process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。