arXiv:2603.17948cs.CVcs.AI2026-03

用分层网格结构实现视频的无损导航,计算量随时长对数增长。

VideoAtlas: Navigating Long-Form Video in Logarithmic Compute

  • 构建分层网格视频表示,支持无损递归缩放和统一视觉记忆。
  • 计算量仅随视频时长对数级增长,缓存命中率达30%-60%。
  • 适合需要长视频理解且关注计算效率的研究者与开发者。

将语言模型扩展至视频面临两大挑战:表示层面依赖有损近似,长上下文处理中基于字幕或代理的流程将视频压缩为文本,损失视觉细节。为此,我们提出 extbf{VideoAtlas}——一种任务无关的环境,以分层网格形式无损、可导航、可扩展地表示视频,无需字幕或预处理。视频概览一目了然,任意区域可递归放大,统一视觉表征贯穿视频、中间探索与代理记忆,全程避免有损文本转换。该结构确保访问深度仅随视频长度对数增长。针对长上下文问题,递归语言模型(RLMs)提供了强大方案,但其在视觉领域应用需结构化环境支持, extbf{VideoAtlas} 正提供此能力。将 extbf{VideoAtlas} 视为马尔可夫决策过程,催生 Video-RLM:一种并行主-工架构,主节点协调全局探索,工作节点并发深入分配区域,积累无损视觉证据。我们验证三大发现:(1) 计算量随视频时长呈对数增长,且因网格结构复用,多模态缓存命中率提升30%-60%;(2) 环境预算机制通过限制最大探索深度,提供可解释的计算-精度超参数;(3) 出现自适应计算分配,随问题粒度动态调整。从1小时到10小时基准测试中,Video-RLM 保持最强时长鲁棒性,准确率下降最小,证明结构化环境导航是视频理解可行且可扩展的新范式。

原文摘要 · Abstract (English)

Extending language models to video introduces two challenges: representation, where existing methods rely on lossy approximations, and long-context, where caption- or agent-based pipelines collapse video into text and lose visual fidelity. To overcome this, we introduce \textbf{VideoAtlas}, a task-agnostic environment to represent video as a hierarchical grid that is simultaneously lossless, navigable, scalable, caption- and preprocessing-free. An overview of the video is available at a glance, and any region can be recursively zoomed into, with the same visual representation used uniformly for the video, intermediate investigations, and the agent's memory, eliminating lossy text conversion end-to-end. This hierarchical structure ensures access depth grows only logarithmically with video length. For long-context, Recursive Language Models (RLMs) recently offered a powerful solution for long text, but extending them to visual domain requires a structured environment to recurse into, which \textbf{VideoAtlas} provides. \textbf{VideoAtlas} as a Markov Decision Process unlocks Video-RLM: a parallel Master-Worker architecture where a Master coordinates global exploration while Workers concurrently drill into assigned regions to accumulate lossless visual evidence. We demonstrate three key findings: (1)~logarithmic compute growth with video duration, further amplified by a 30-60\% multimodal cache hit rate arising from the grid's structural reuse. (2)~environment budgeting, where bounding the maximum exploration depth provides a principled compute-accuracy hyperparameter. (3)~emergent adaptive compute allocation that scales with question granularity. When scaling from 1-hour to 10-hour benchmarks, Video-RLM remains the most duration-robust method with minimal accuracy degradation, demonstrating that structured environment navigation is a viable and scalable paradigm for video understanding.

视频理解长视频分层结构计算效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。