让大模型边读边思考,动态分配计算资源。
AdaSR: Adaptive Streaming Reasoning with Hierarchical Relative Policy Optimization

- 分阶段优化推理策略,按需决定何时思考、思考多久。
- 在多项任务上同时提升准确率与计算效率,延迟更低。
- 适合实时流式处理场景,如视频/语音理解等应用。
大型推理模型通常采用先读再思考的模式:完整接收输入后,在静态上下文中进行推理并生成答案。然而,许多现实场景(如音视频流)具有动态性,信息持续到达,模型必须在部分观测下进行推理、更新和响应。现有流式推理方法虽支持边读边思考,但大多依赖预构建轨迹的监督模仿,灵活性受限。本文提出AdaSR,一种自适应流式推理框架,使模型能在输入流过程中进行推理,并在流结束时进行最终反思,学习何时思考及各阶段应分配多少计算量。为优化这一分层推理过程,我们引入层次化相对策略优化(HRPO),将策略优化分解为流式推理与深度推理两个阶段,提供更精细的优势分配,而非对所有标记统一分配单一序列级优势。HRPO融合格式、准确率与自适应思考奖励,以确保有效推理协议、保持最终任务性能,并鼓励低延迟计算分配。实验表明,相比监督微调基线,AdaSR在推理准确率、计算效率与流式延迟之间实现了更优平衡。代码已开源:https://github.com/EIT-NLP/StreamingLLM/tree/main/AdaSR。
原文摘要 · Abstract (English)
Large reasoning models typically follow a read-then-think paradigm: they observe the complete input, reason over a static context, and then produce the answer. Yet many real-world scenarios are inherently dynamic, such as audio and video stream, where information arrives as a continuous stream and models must reason, update, and respond under partial observations. Recent streaming reasoning methods allow models to think while reading, but they largely rely on supervised imitation of pre-constructed trajectories, which limits their flexibility. In this paper, we propose AdaSR, an adaptive streaming reasoning framework that enables models to reason during input streaming and perform final deliberation once the stream is complete, learning when to think, and how much computation to allocate across different stages. To optimize this hierarchical reasoning process, we introduce Hierarchical Relative Policy Optimization (HRPO), which decomposes policy optimization into streaming reasoning and deep reasoning phases, providing more fine-grained advantage assignment instead of uniformly distributing a single sequence-level advantage over all tokens. HRPO integrates format, accuracy, and adaptive thinking rewards to enforce valid reasoning protocols, preserve final task performance, and encourage latency-aware computation allocation. Experiments show that AdaSR achieves a better balance among reasoning accuracy, computational efficiency, and streaming latency compared with supervised fine-tuning baseline. We release our code at https://github.com/EIT-NLP/StreamingLLM/tree/main/AdaSR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。