arXiv:2606.21734cs.CVcs.AI2026-06

将视频理解拆解为分层探查,让模型像编程一样逐步推理。

HPP: Hierarchical Programmatic Probing for Long Video Understanding by Decoupling Perception and Reasoning

论文配图:HPP: Hierarchical Programmatic Probing for Long Video Understanding by Decoupling Perception and Reasoning
图 1 · 摘自论文原文
  • 用可编程环境分步探查视频,分离感知与推理过程。
  • 在LongVideoBench上提升显著,多任务表现优于端到端模型。
  • 适合需要长程推理的复杂视频分析任务,如自动驾驶、教育视频。

理解长视频需精细感知与多步、高阶的时间推理能力。当前视觉语言模型(VLM)在单次前向传播中同时完成感知与规划,但受限于大语言模型(LLM)在隐式表示中发现并执行多步策略的能力。为此,我们提出分层程序化探查(HPP)框架,通过将语义感知与高层时间推理解耦,将长视频理解重构为对分层分割视频的迭代、程序化探索。具体而言,一个具备编码能力的LLM在交互式编码环境中制定并执行多步策略,按需调用VLM进行局部感知。为实现长视频上的高效探查,引入三个组件:基于信息密度的分层分割、后期交互语义检索,以及从粗到细的时间定位结构化探查函数。在LongVideoBench验证中,该方法在细粒度感知与长程关系推理任务上均取得显著提升;在EgoSchema、VideoMME和MLVU等多样基准上也表现优异,证明其广泛有效性。

原文摘要 · Abstract (English)

Understanding long videos requires fine-grained perception and multi-step, higher-order reasoning over complex, long-range spatio-temporal dynamics. Vision-language models (VLMs) encode video frames into visual tokens and attempt to perform both perception and multi-step planning latently, within a single forward pass. This coupled formulation, however, is bottlenecked by the LLM's limited capacity to discover and execute multi-step strategies in its latent representations. To address this bottleneck, we propose Hierarchical Programmatic Probing (HPP), a framework that decouples semantic perception from higher-order temporal reasoning by reformulating long video understanding as iterative, programmatic exploration of a hierarchically segmented video. Specifically, a coding-capable LLM plans and executes a multi-step strategy in an interactive coding environment, probing the video for information and invoking a VLM for localized perception on demand. To make probing tractable over long videos, we introduce three components: information-density-aware hierarchical segmentation, late-interaction semantic retrieval, and structured probing functions for coarse-to-fine temporal localization. We validate HPP on LongVideoBench, which requires both fine-grained perception and long-range relational reasoning, and show that decoupling the two via iterative programmatic probing yields substantial gains. Further results on EgoSchema, VideoMME, and MLVU demonstrate the effectiveness of our approach across diverse long-video benchmarks.

视频理解分层推理程序化探查

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。