针对第一人称视频理解难题，提出端到端优化框架提升问答准确率。
Optimizing Multimodal LLMs for Egocentric Video Understanding: A Solution for the HD-EPIC VQA Challenge
- 通过查询预处理与领域微调增强模型理解力
- 引入时间链式思维提示，实现多步推理，准确率达41.6%
- 适合关注视频理解、多模态模型优化的研究者
多模态大语言模型在复杂视频问答基准（如HD-EPIC VQA）上表现不佳，主要因查询模糊、长时序推理能力弱及输出不统一。本文提出一个集成框架：包括查询/选项预处理、领域特定的Qwen2.5-VL微调、一种新型时间链式思维（T-CoT）提示以支持多步推理，以及鲁棒后处理。该系统在HD-EPIC VQA上达到41.6%的准确率，凸显了在高难度视频理解任务中需进行整体流程优化的必要性。代码与微调模型已公开于https://github.com/YoungSeng/Egocentric-Co-Pilot。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) struggle with complex video QA benchmarks like HD-EPIC VQA due to ambiguous queries/options, poor long-range temporal reasoning, and non-standardized outputs. We propose a framework integrating query/choice pre-processing, domain-specific Qwen2.5-VL fine-tuning, a novel Temporal Chain-of-Thought (T-CoT) prompting for multi-step reasoning, and robust post-processing. This system achieves 41.6% accuracy on HD-EPIC VQA, highlighting the need for holistic pipeline optimization in demanding video understanding. Our code, fine-tuned models are available at https://github.com/YoungSeng/Egocentric-Co-Pilot.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。