让视频模型像人一样反复看文字细节，提升细粒度理解能力。
Video-R4: Reinforcing Text-Rich Video Reasoning with Visual Rumination
- 引入视觉沉思机制，迭代选择帧、缩放区域、重编码像素并更新推理状态。
- 在M4-ViteVQA上达最优性能，30k条强化学习轨迹支持训练。
- 适用于文档、幻灯片等多场景，适合需要精细图文理解的场景。
理解富含文字的视频需反复观察短暂出现的文字线索，但多数视频问答模型仅单次扫描固定帧，导致幻觉和细粒度证据识别失败。受人类暂停、放大、重读关键区域的启发，我们提出Video-R4（通过视觉沉思强化文本丰富视频推理），一种具备视觉沉思能力的视频推理大模型：通过迭代选择帧、缩放信息区域、重新编码像素并更新推理状态。我们构建了两个含可执行沉思轨迹的数据集：Video-R4-CoT-17k用于监督学习，Video-R4-RL-30k用于强化学习。提出多阶段沉思学习框架，逐步微调7B规模的大模型，通过SFT与基于GRPO的强化学习学会基础与组合视觉操作。Video-R4-7B在M4-ViteVQA上达到当前最佳表现，并可泛化至多页文档问答、幻灯片问答及通用视频问答，证明迭代沉思是像素级多模态推理的有效范式。
原文摘要 · Abstract (English)
Understanding text-rich videos requires reading small, transient textual cues that often demand repeated inspection. Yet most video QA models rely on single-pass perception over fixed frames, leading to hallucinations and failures on fine-grained evidence. Inspired by how humans pause, zoom, and re-read critical regions, we introduce Video-R4 (Reinforcing Text-Rich Video Reasoning with Visual Rumination), a video reasoning LMM that performs visual rumination: iteratively selecting frames, zooming into informative regions, re-encoding retrieved pixels, and updating its reasoning state. We construct two datasets with executable rumination trajectories: Video-R4-CoT-17k for supervised practice and Video-R4-RL-30k for reinforcement learning. We propose a multi-stage rumination learning framework that progressively finetunes a 7B LMM to learn atomic and mixing visual operations via SFT and GRPO-based RL. Video-R4-7B achieves state-of-the-art results on M4-ViteVQA and further generalizes to multi-page document QA, slides QA, and generic video QA, demonstrating that iterative rumination is an effective paradigm for pixel-grounded multimodal reasoning. Project Page: https://yunlong10.github.io/Video-R4/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。