评测视频模型在长时序理解上的能力，提出首个小时级视频问答基准。
Perception Test 2024: Challenge Summary and a Novel Hour-Long VideoQA Benchmark
- 设立七项跨模态任务，涵盖跟踪、定位与问答等多类感知任务
- 引入1小时视频问答新基准，测试模型对长时序上下文的理解能力
- 适合关注视频理解、多模态模型评估的研究者参考
继2023年成功举办后，我们于2024年欧洲计算机视觉会议（ECCV）期间组织了第二届感知测试挑战赛，作为半天工作坊，旨在通过感知测试基准评估当前最先进的视频模型，并衡量自去年以来的进展。今年挑战赛共设七个赛道（较去年增加一项），覆盖低层与高层任务，包含语言与非语言接口，涉及视频、音频与文本模态；新增赛道聚焦小时级视频理解，提出全新1小时视频问答基准1h-walk VQA。各赛道任务包括：目标跟踪、点跟踪、时间动作定位、时间声音定位、多项选择视频问答、图像定位视频问答及小时级视频问答。本文总结挑战赛任务与结果，并详细介绍新型1小时视频问答基准1h-walk VQA。
原文摘要 · Abstract (English)
Following the successful 2023 edition, we organised the Second Perception Test challenge as a half-day workshop alongside the IEEE/CVF European Conference on Computer Vision (ECCV) 2024, with the goal of benchmarking state-of-the-art video models and measuring the progress since last year using the Perception Test benchmark. This year, the challenge had seven tracks (up from six last year) and covered low-level and high-level tasks, with language and non-language interfaces, across video, audio, and text modalities; the additional track covered hour-long video understanding and introduced a novel video QA benchmark 1h-walk VQA. Overall, the tasks in the different tracks were: object tracking, point tracking, temporal action localisation, temporal sound localisation, multiple-choice video question-answering, grounded video question-answering, and hour-long video question-answering. We summarise in this report the challenge tasks and results, and introduce in detail the novel hour-long video QA benchmark 1h-walk VQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。