arXiv:2510.17238cs.CL2025-10被引 14

让大模型读一段话就立刻开始思考,比传统方法快60%以上。

StreamingThinker: Large Language Models Can Think While Reading

  • 输入流式到达时即开始逐步推理,不再等全部输入完成。
  • 在数学、逻辑和问答任务上性能接近批量推理,但延迟降低60%以上。
  • 适合需要实时响应的场景,如在线对话或动态信息处理。

大型语言模型(LLM)在链式思维(CoT)推理中表现出色,但现有范式需等待完整输入后才开始推理,导致延迟高且弱化对早期信息的关注。受人类阅读时边读边思考的启发,我们首次提出一种‘流式思考’范式,使推理随输入顺序展开,并在阅读完成后可调整深度。我们构建了StreamingThinker框架,通过流式CoT生成、流式约束训练与流式并行推理实现该范式。其采用带质量控制的流式推理单元,利用流式注意力掩码与位置编码保持顺序一致性,并通过解耦输入编码与推理生成的并行键值缓存,确保对齐与真正并发。我们在Qwen3模型族上评估了该方法,在数学推理、逻辑推理和基于上下文的问答任务中,性能接近批处理推理,同时将推理启动前的令牌等待时间减少80%,最终答案生成的时间延迟降低超过60%,验证了流式推理的有效性。代码已公开于https://github.com/EIT-NLP/StreamingLLM/tree/main/StreamingThinker。

原文摘要 · Abstract (English)

Large language models (LLMs) have demonstrated remarkable capabilities in chain of thought (CoT) reasoning. However, the current LLM reasoning paradigm initiates thinking only after the entire input is available, which introduces unnecessary latency and weakens attention to earlier information in dynamic scenarios. Inspired by human cognition of thinking while reading, we first design a \textit{\textbf{streaming thinking}} paradigm for LLMs, where reasoning unfolds in the order of input and further adjusts its depth once reading is complete. We instantiate this paradigm with \textit{StreamingThinker}, a framework that enables LLMs to think while reading through the integration of streaming CoT generation, streaming-constraint training, and streaming parallel inference. Specifically, StreamingThinker employs streaming reasoning units with quality control for CoT generation, enforces order-preserving reasoning through streaming attention masks and position encoding, and leverages parallel KV caches that decouple input encoding from reasoning generation, thereby ensuring alignment and enabling true concurrency. We evaluate StreamingThinker on the Qwen3 model family across math reasoning, logical reasoning, and context-based QA reasoning tasks. Experimental results show that the StreamingThinker preserves performance comparable to batch thinking, while yielding an 80\% reduction in token waiting before the onset of reasoning and a more than 60\% reduction in time-level latency for producing the final answer, demonstrating the effectiveness of the streaming paradigm for LLM reasoning. Code is publicly available at https://github.com/EIT-NLP/StreamingLLM/tree/main/StreamingThinker.

大模型推理流式处理思维链

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。