让视频模型学会在信息不全时主动决策,而非等证据出现才反应。
StreamPro: From Reactive Perception to Proactive Decision-Making in Streaming Video

- 分两阶段训练:先用新损失缓解信号失衡,再用多粒度奖励优化决策时机与正确性。
- 在新基准上主动决策准确率达41.5,远超此前最佳的10.4。
- 适合研究视频理解中实时决策、主动感知的科研与工程人员。
主动流式视频理解要求模型持续处理视频流并决定何时响应,而不仅决定如何响应。这自然引出部分观测下的决策问题,模型需在早期预测与充分证据间权衡。然而,现有基准大多遵循“先见后答”范式,仅在显式证据出现后才触发响应,将主动推理简化为延迟感知,无法评估模型在不完备观察下做出及时可靠决策的能力。此外,训练主动模型因流轨迹中静默与响应信号严重失衡,且需联合优化响应正确性与时机,面临巨大挑战。为此,我们提出 StreamPro-Bench 基准,从感知理解、时间推理和主动代理三个互补视角评估流式模型,其中最后一项衡量模型在部分观测下做出早期且可靠的决策能力。我们进一步提出 StreamPro 两阶段训练框架:首先引入 CB-Stream Loss 缓解监督微调中的严重监督失衡;随后采用具有多粒度奖励设计的组相对策略优化(GRPO),包含回合级与轨迹级奖励。实验表明,StreamPro 显著提升主动性能,在 StreamPro-Bench 上达到 41.5 分,大幅优于此前最佳(10.4),同时在真实流式基准 StreamingBench-RTVU 上保持优异表现,达 78.9 分。
原文摘要 · Abstract (English)
Proactive streaming video understanding requires models to continuously process video streams and decide when to respond, rather than merely what to respond. This naturally introduces a decision-making problem under partial observations, where models must balance early prediction against sufficient evidence. However, existing benchmarks largely follow a "see-then-answer" paradigm, where responses are triggered only after explicit evidence appears, effectively reducing proactive reasoning to delayed perception. As a result, they fail to evaluate a model's ability to make timely and reliable decisions under incomplete observations. Moreover, training proactive models is inherently challenging due to the extreme imbalance between silence and response signals in streaming trajectories, as well as the need to jointly optimize response correctness and timing. To address these challenges, we introduce StreamPro-Bench, a new benchmark that evaluates streaming models from three complementary perspectives: Perception Understanding, Temporal Reasoning, and Proactive Agency, where the last measures a model's ability to make early yet reliable decisions under partial observations. We further propose StreamPro, a two-stage training framework for proactive learning. First, we introduce CB-Stream Loss to mitigate the severe supervision imbalance during supervised fine-tuning (SFT). Then, we apply Group Relative Policy Optimization (GRPO) with a multi-grained reward design that involves both turn-level and trajectory-level rewards. Experiments show that StreamPro significantly improves proactive performance. On StreamPro-Bench, it achieves 41.5, substantially outperforming the previous best (10.4), while also maintaining strong performance on real-time streaming benchmarks, achieving 78.9 on StreamingBench-RTVU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。