arXiv:2605.21443cs.CVcs.AI2026-05

提出新基准TempGlitch,评估视觉语言模型对游戏视频时序漏洞的检测能力。

TempGlitch: Evaluating Vision-Language Models for Temporal Glitch Detection in Gameplay Videos

论文配图:TempGlitch: Evaluating Vision-Language Models for Temporal Glitch Detection in Gameplay Videos
图 1 · 摘自论文原文
  • 构建五类时序漏洞数据集,对比有无漏洞的视频对进行评测。
  • 12个VLM模型在该任务上表现接近随机,多数误判严重。
  • 适合研究时序推理、游戏自动化测试和鲁棒视频理解的学者。

视觉语言模型(VLMs)在游戏质量保证中日益受到关注,尤其用于检测游戏玩法中的漏洞。现有评估大多将漏洞视为静态视觉异常,仅从单帧判断。我们指出这种设定忽略了关键差异:部分漏洞是空间性的,可在孤立帧中显现;而另一些是时序性的,需通过有序帧间的动态变化才能识别。初步研究表明,时序漏洞对VLM的检测难度显著高于空间漏洞。为系统评估这一被忽视的场景,我们引入TempGlitch——一个受控的游戏玩法视频基准,涵盖五类时序漏洞,每类样本均衡,并配有对应的无漏洞视频,支持可靠的二元评估。我们在多种帧采样设置下评估了12个专有及开源权重的VLM。结果表明,当前模型在TempGlitch上的表现接近随机水平,常呈现过度保守(漏检)或过度敏感(误报)行为。更密集的帧采样和更大模型规模也无法稳定提升性能。TempGlitch为时序推理、鲁棒游戏理解与自动化漏洞检测提供了专注的测试平台。代码与数据见项目官网。

原文摘要 · Abstract (English)

Vision-language models (VLMs) are increasingly being explored for video game quality assurance, especially gameplay glitch detection. Most existing evaluations, however, treat glitches as static visual anomalies, asking models to detect failures from a single frame. We argue that this framing misses a key distinction: some glitches are spatial and visible in an isolated frame, whereas others are temporal and become evident only through changes across ordered frames. A preliminary study confirms this gap, showing that temporal glitches are substantially harder for VLMs to detect than spatial ones. To enable systematic evaluation of this underexplored setting, we introduce TempGlitch, a controlled gameplay video benchmark for temporal glitch detection. TempGlitch covers five temporal glitch types with balanced per-category samples, together with paired glitch-free videos that enable reliable binary evaluation. We evaluate 12 proprietary and open-weight VLMs across multiple frame-sampling settings. Our results show that current VLMs remain near chance on TempGlitch, often collapsing into either overly conservative behavior that misses most glitches or overly sensitive behavior that flags clean videos as glitchy. Moreover, denser frame sampling and larger model size do not reliably resolve these failures. TempGlitch provides a focused testbed for temporal reasoning, robust gameplay understanding, and automated glitch detection with VLMs. Code and data are available at the project website.

时序检测游戏测试视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。