让大模型像看视频一样实时推理，提升响应速度与准确性。
Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models
- 提出边看边想的流式推理框架，支持帧到帧实时分析。
- 在多个视频理解任务上显著降低首次输出延迟，性能优于传统方法。
- 适合需要低延迟响应的实时视频分析场景，如监控、直播互动。
大型视觉语言模型（LVLMs）具备强大的思维链（CoT）能力，但现有方法大多假设视频数据全部可用后再进行推理，属于批处理模式，与真实世界视频流中信息逐帧到达的特性不符。为匹配视频数据的流式特征，本文研究了两种流式推理范式。第一种为交错式范式，交替接收帧与生成部分推理，但受限于严格的缓存更新顺序。为此，我们提出 extbf{Think-as-You-See (TaYS)}，一个统一的并行化流式推理框架，实现真正意义上的并发推理。TaYS结合并行化思维链生成、流约束训练及流并行推理，引入时间对齐的推理单元、流式注意力掩码与位置编码，并采用双键值缓存机制，解耦视觉编码与文本推理。我们在Qwen2.5-VL系列模型上评估了多种范式，涵盖事件动态分析、因果推理与主题理解等代表性任务。实验表明，TaYS在各项任务中均优于批处理与交错基线，显著提升推理性能，同时大幅降低首次输出时间（TTFT）与整体推理延迟。结果证明，数据对齐的流式推理可有效提升LVLM在视频理解中的效率与响应性。代码已开源：https://github.com/EIT-NLP/StreamingLLM/tree/main/TaYS
原文摘要 · Abstract (English)
Large Vision Language Models (LVLMs) exhibit strong Chain-of-Thought (CoT) capabilities, yet most existing paradigms assume full-video availability before inference, a batch-style process misaligned with real-world video streams where information arrives sequentially. Motivated by the streaming nature of video data, we investigate two streaming reasoning paradigms for LVLMs. The first, an interleaved paradigm, alternates between receiving frames and producing partial reasoning but remains constrained by strictly ordered cache updates. To better match streaming inputs, we propose \textbf{Think-as-You-See (TaYS)}, a unified framework enabling true concurrent reasoning. TaYS integrates parallelized CoT generation, stream-constrained training, and stream-parallel inference. It further employs temporally aligned reasoning units, streaming attention masks and positional encodings, and a dual KV-cache that decouples visual encoding from textual reasoning. We evaluate all paradigms on the Qwen2.5-VL family across representative video CoT tasks, including event dynamics analysis, causal reasoning, and thematic understanding. Experiments show that TaYS consistently outperforms both batch and interleaved baselines, improving reasoning performance while substantially reducing time-to-first-token (TTFT) and overall reasoning delay. These results demonstrate the effectiveness of data-aligned streaming reasoning in enabling efficient and responsive video understanding for LVLMs. We release our code at https://github.com/EIT-NLP/StreamingLLM/tree/main/TaYS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。