让视频问答更懂问题，精准定位关键帧
Question-Aware Gaussian Experts for Audio-Visual Question Answering
- 用高斯模型动态聚焦与问题相关的连续/非连续帧
- 在多个基准上达到当前最好性能，超越现有方法
- 适合需要细粒度时序理解的多模态问答研究
音频-视觉问答（AVQA）不仅需要基于问题的多模态推理，还需精确的时序定位以捕捉细微动态。现有方法多隐式使用问题信息，难以关注特定细节；且普遍采用均匀采样，易遗漏关键帧。虽有Top-K采样缓解此问题，但其离散性仍忽略细粒度时序特征。本文提出QA-TIGER框架，显式融合问题信息并建模连续时序动态。核心思想是利用高斯模型自适应聚焦相关帧，结合问题信息进行渐进式优化。通过混合专家（MoE）灵活实现多个高斯模型，激活与问题匹配的时序专家。在多个AVQA基准上的大量实验表明，该方法持续取得最先进性能。代码已开源。
原文摘要 · Abstract (English)
Audio-Visual Question Answering (AVQA) requires not only question-based multimodal reasoning but also precise temporal grounding to capture subtle dynamics for accurate prediction. However, existing methods mainly use question information implicitly, limiting focus on question-specific details. Furthermore, most studies rely on uniform frame sampling, which can miss key question-relevant frames. Although recent Top-K frame selection methods aim to address this, their discrete nature still overlooks fine-grained temporal details. This paper proposes QA-TIGER, a novel framework that explicitly incorporates question information and models continuous temporal dynamics. Our key idea is to use Gaussian-based modeling to adaptively focus on both consecutive and non-consecutive frames based on the question, while explicitly injecting question information and applying progressive refinement. We leverage a Mixture of Experts (MoE) to flexibly implement multiple Gaussian models, activating temporal experts specifically tailored to the question. Extensive experiments on multiple AVQA benchmarks show that QA-TIGER consistently achieves state-of-the-art performance. Code is available at https://aim-skku.github.io/QA-TIGER/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。