提出新任务，让模型学会拒绝无关问题，避免误检。
Moment of Untruth: Dealing with Negative Queries in Video Moment Retrieval
- 设计新任务NA-VMR，区分相关与无关查询。
- 模型对无关问题拒绝率达98.4%，且不牺牲定位精度。
- 适合想提升视觉语言模型鲁棒性的研究者。
视频时刻检索是评估视觉-语言模型性能的常见任务，需从查询句子中定位视频中的起止时间。当前设定假设查询时刻必存在于视频中，导致提供无关句子时产生假阳性结果。本文提出负样本感知视频时刻检索（NA-VMR）任务，同时关注时刻定位准确率与负查询拒绝准确率。我们区分了域内与域外负样本，并为两个主流数据集QVHighlights和Charades-STA构建新评估基准。分析现有最先进方法在该任务上的适应能力，提出UniVTG-NA，即针对NA-VMR优化的UniVTG改进版。UniVTG-NA在保持召回率下降不超过3.87%的前提下，实现平均98.4%的负样本拒绝率。代码与数据划分已开源。
原文摘要 · Abstract (English)
Video Moment Retrieval is a common task to evaluate the performance of visual-language models - it involves localising start and end times of moments in videos from query sentences. The current task formulation assumes that the queried moment is present in the video, resulting in false positive moment predictions when irrelevant query sentences are provided. In this paper we propose the task of Negative-Aware Video Moment Retrieval (NA-VMR), which considers both moment retrieval accuracy and negative query rejection accuracy. We make the distinction between In-Domain and Out-of-Domain negative queries and provide new evaluation benchmarks for two popular video moment retrieval datasets: QVHighlights and Charades-STA. We analyse the ability of current SOTA video moment retrieval approaches to adapt to Negative-Aware Video Moment Retrieval and propose UniVTG-NA, an adaptation of UniVTG designed to tackle NA-VMR. UniVTG-NA achieves high negative rejection accuracy (avg. $98.4\%$) scores while retaining moment retrieval scores to within $3.87\%$ Recall@1. Dataset splits and code are available at https://github.com/keflanagan/MomentofUntruth
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。