用大模型重排序提升视频定位精度，双赛道夺冠
OSGNet with MLLM Reranking @ Ego4D Episodic Memory Challenge 2026
- 先用OSGNet生成候选片段，再用MLLM重排序选最优
- 在自然语言和目标步骤任务中均获第一名
- 结合大模型推理与高效候选召回，适合长视频定位
本文报告了我们在CVPR 2026年Ego4D情景记忆挑战赛中自然语言查询与目标步骤赛道的冠军方案。两个赛道均需从长时未剪辑的第一人称视频中准确定位时间片段。我们提出一种基于重排序的框架，有效利用多模态大语言模型（MLLM）强大的视频-语言推理能力，同时保持传统定位流程的效率与候选召回率。具体而言，首先通过现有定位模型OSGNet获取一组候选片段，随后使用MLLM筛选最匹配查询的片段，从而优化最终预测。该方法在两个赛道中均取得第一名。代码已公开于https://github.com/iLearn-Lab/CVPR25-OSGNet。
原文摘要 · Abstract (English)
In this report, we present our champion solutions for the Natural Language Queries and GoalStep tracks of the Ego4D Episodic Memory Challenge at CVPR 2026. Both tracks require accurately localizing temporal segments from long untrimmed egocentric videos. To address these tasks, we propose a reranking-based framework that effectively leverages the strong video-language reasoning capability of multimodal large language model (MLLM) while preserving the efficiency and candidate recall of conventional localization pipelines. Specifically, we first obtain a set of candidate segments from existing localization model OSGNet, and then employ MLLM to select the segment that best matches the given query, thereby refining the final prediction. Ultimately, our method achieved first place in both the Natural Language Queries and GoalStep tracks. Our code can be found at https://github.com/iLearn-Lab/CVPR25-OSGNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。