arXiv:2603.22706cs.CVcs.SE2026-03

测试19738帧游戏视频,发现现成视觉语言模型检测视觉漏洞能力有限。

How Far Can VLMs Go for Visual Bug Detection? Studying 19,738 Keyframes from 41 Hours of Gameplay Videos

  • 用单轮提示评估19738帧游戏画面中的视觉错误
  • 精确率0.50,准确率0.72,增强策略提升微弱
  • 适合关注自动化游戏质检的工程团队参考

基于视频的游戏质量保证(QA)耗时且易出错,但对评估游戏稳定性与视觉正确性至关重要。视觉语言模型(VLMs)具备通用视觉推理能力,被认为可直接从视频帧中检测视觉缺陷。已有基准显示,VLM在精选数据集上表现良好。本研究使用工业级游戏质量保证视频,在总计100段、41小时、19,738个关键帧的视频中评估VLM实际表现。从单提示基线开始,模型取得0.50的精确率和0.72的准确率。我们进一步测试两种无需微调的增强策略:(1)使用次级判断模型重新评估输出;(2)通过检索历史缺陷报告进行元数据增强提示。结果显示,两种策略仅带来微弱提升,却增加计算开销与输出波动。结果表明,现成的VLM已能识别部分视觉缺陷,但未来进展需依赖文本与视觉异常检测分离的混合方法。

原文摘要 · Abstract (English)

Video-based quality assurance (QA) for long-form gameplay video is labor-intensive and error-prone, yet valuable for assessing game stability and visual correctness over extended play sessions. Vision language models (VLMs) promise general-purpose visual reasoning capabilities and thus appear attractive for detecting visual bugs directly from video frames. Recent benchmarks suggest that VLMs can achieve promising results in detecting visual glitches on curated datasets. Building on these findings, we conduct a real-world study using industrial QA gameplay videos to evaluate how well VLMs perform in practical scenarios. Our study samples keyframes from long gameplay videos and asks a VLM whether each keyframe contains a bug. Starting from a single-prompt baseline, the model achieves a precision of 0.50 and an accuracy of 0.72. We then examine two common enhancement strategies used to improve VLM performance without fine-tuning: (1) a secondary judge model that re-evaluates VLM outputs, and (2) metadata-augmented prompting through the retrieval of prior bug reports. Across \textbf{100 videos} totaling \textbf{41 hours} and \textbf{19,738 keyframes}, these strategies provide only marginal improvements over the simple baseline, while introducing additional computational cost and output variance. Our findings indicate that off-the-shelf VLMs are already capable of detecting a certain range of visual bugs in QA gameplay videos, but further progress likely requires hybrid approaches that better separate textual and visual anomaly detection.

视觉检测游戏质检VLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。