构建首个需跨帧融合文本的视频文字问答数据集，解决模型时间感知短板。
ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answering
- 通过带时间约束的思维链标注，确保问题必须跨帧理解
- 提出两阶段训练框架，使模型能生成带时序意识的推理链
- 在新数据集上提升ROUGE-L达6.3%，适合视频理解研究者
尽管多模态大模型取得显著进展，现有模型在视频文本理解上仍存在局限，尤其当语义依赖于多帧间分布的文本线索整合时。这一感知挑战与静态图像文本理解本质不同，但当前数据集大多只需单帧即可作答，无法反映真实视频文本理解需求。为此，我们构建了大规模视频文本问答数据集ViTexQA，并提出FrameThinker以实现鲁棒的多帧时序推理。通过受控质量的思维链（CoT）标注流程并引入时间约束，所有问答对均要求跨帧文本融合才能解答，强制模型具备真正的时序依赖性。FrameThinker采用两阶段训练：先通过CoT引导的监督微调（SFT）生成帧感知推理链，再通过基于多帧一致性的强化学习（RL）优化。评估显示，该方法在ViTexQA上超越现有最优基线，ROUGE-L提升6.3%。
原文摘要 · Abstract (English)
Despite remarkable progress in multimodal understanding, current MLLMs still exhibit limitations in video text understanding, particularly when semantics emerge through the integration of temporally distributed textual cues across multiple frames. This perception challenge fundamentally differs from static image text understanding, yet existing datasets fail to capture: the vast majority of questions remain answerable from single frames, inadequately reflecting real-world video text comprehension demands. To address this, we present ViTexQA, a large-scale video-text QA dataset, and FrameThinker for robust multi-frame temporal reasoning. We build ViTexQA via a quality-controlled Chain-of-Thought (CoT) annotation pipeline boosted with temporal constraints; all its QA pairs demand cross-frame text fusion to solve, enforcing true temporal reliance. FrameThinker adopts two-stage training for explicit temporal modeling: CoT-Guided Supervised Fine-Tuning (SFT) generates frame-aware reasoning chains, followed by Temporally-grounded Reinforcement Learning (RL) optimized with multi-frame coherence rewards. Evaluations show our method outperforms SOTA baselines on ViTexQA, lifting ROUGE-L by 6.3%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。