提出视频模型记忆能力量化模型，揭示帧预算与时间距离的准确率关系。
How Well Can Your Video Model Remember? Measuring Memory-Budget Trade-offs in Long Video Understanding
- 构建对数线性模型，预测不同帧预算下长时视频理解准确率变化
- 发现最优流式模型在1000秒距离下帧增益效率是基线模型的7.4倍
- 提出可诊断流式视频模型性能的预算效率指标α(D)，适合模型优化者
我们提出一个紧凑的实证模型，量化在长视频理解中，回答准确率随帧预算B和时间距离D的退化情况——即利用总帧数的分数B来回忆过去D秒内容的表现。长视频模型受限于严格预算，但此前缺乏预测准确率随预算缩减和事件远移如何变化的框架。我们在约15.5万次二分类预测上拟合加权最小二乘模型，涵盖十种模型和三种采样策略，推导出一条规律：对数精度随对数预算线性变化，其指数α(D)随距离对数线性衰减。该指数捕捉了在距离D处额外帧的边际价值。该规律在不同模型中达到单元级加权R²为0.05–0.75。显著的是，在D=1000秒时，最优流式模型与基线模型的预算有效性相差约7.4倍。STREAMINGVLM在该距离下的α(1000)=1.26（95%置信区间[1.06, 1.58]），表示预算增加十倍可大幅提升长距离准确率；而最佳Qwen3-VL基线模型仅达α(1000)=0.17（置信区间[0.04, 0.34]）。在准确率空间中，预算提升十倍时，STREAMINGVLM在1000秒距离下准确率提升29个百分点，而基线模型仅提升4个百分点。采样策略显示模型依赖的权衡：随机采样虽基础敏感度更高，但距离衰减更剧烈。我们证明α(D)可用于指导合理的预算分配，甚至导致模型排名反转，建议将其作为流式视频模型的诊断指标。
原文摘要 · Abstract (English)
We introduce a compact empirical model that quantifies how answer accuracy degrades as a function of frame budget B and temporal distance D in long video understanding -- analyzing performance when recalling content from D seconds in the past using a fraction B of total frames. Long-form models operate under strict budgets, yet no prior framework predicts how accuracy degrades as B shrinks and events recede. We fit a weighted least-squares model on ~155,000 binary predictions across ten models and three sampling strategies, deriving a law where logit-accuracy scales linearly in log-budget with a distance-dependent exponent that decays log-linearly with distance. This budget exponent α(D) captures the marginal value of extra frames at distance D. The law achieves cell-level weighted R^2 = 0.05-0.75 across models. Notably, budget effectiveness at D = 1000 s differs by \approx 7.4\times between the best streaming and base models. STREAMINGVLM achieves α(1000) = 1.26 (95% CI: [1.06, 1.58]), meaning a tenfold budget increase substantially improves long-distance accuracy, while the best Qwen3-VL base model reaches only α(1000) = 0.17 (CI: [0.04, 0.34]). In accuracy space, a 10\times budget increase at D = 1000 s yields +29 percentage points for STREAMINGVLM versus +4 pp for the base model. Sampling strategies show model-dependent trade-offs: random sampling yields higher base sensitivity but steeper distance decay. We demonstrate how α(D) enables principled budget allocation, including a model-ranking reversal at long distance, and propose it as a diagnostic metric for streaming video models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。