无需训练的智能体框架，实现多视角长视频的复杂推理。
3rd Place at CVPR 2026 CASTLE Challenge: Agentic Multi-View Long-Context Video Understanding via Hierarchical Knowledge Graph Retrieval
- 构建分层知识图谱，支持多跳关系推理。
- 在600小时多源视频上实现高零样本推理准确率。
- 适合需要长时序理解与跨视图分析的任务场景。
本文介绍了我们在CVPR 2026 EgoVis研讨会举办的CASTLE 2026挑战赛中获得全球第三名的解决方案。该挑战要求参与者对超过600小时、由15个第一人称与第三人称摄像头同步采集的多模态视频流，回答高度复杂的视觉、时空和语言问题，包括视觉计数、动作定位、多视角跟踪及说话人时间推理等任务。为应对极端规模与长上下文需求，我们提出一种无需训练的智能体框架，专注于长视频理解。框架包含两个核心组件：一是将静态与动态实体、其时间关系及交叉事件映射为视频知识图谱，支持多跳关系推理；二是自适应的分层检索与索引智能体工作流，用于分解并解决复杂查询。实验表明，该框架在长上下文多视角视频流上实现了高零样本推理准确率。代码将公开于 https://github.com/RaghadKhaled/CASTLE-Challenge-Framework。
原文摘要 · Abstract (English)
This paper presents our winning methodology for the CASTLE 2026 Challenge at the CVPR 2026 EgoVis Workshop, where our team secured third place globally. The challenge tasks participants with answering highly complex visual, spatiotemporal, and verbal questions, including visual counting, action localization, multi-view tracking and speaker temporal reasoning, within massive, multimodal video streams. The underlying dataset consists of over 600 hours synchronized footage captured by 15 ego and exo camera sources. To tackle the extreme scale and long-context demands of this environment, we introduce a training-free agentic framework optimized for long-form video understanding. Our framework introduces two core architectural components: i) a Video Knowledge Graph that maps static and dynamic entities, their temporal relationships, and intersecting events to enable multi-hop relational reasoning, and ii) an adaptive agentic workflow that resolves complex queries through a hierarchical retrieval and indexing. Empirical results demonstrate that our framework achieves high zero-shot reasoning accuracy on long-context multi-view streams. Our code will be released at https://github.com/RaghadKhaled/CASTLE-Challenge-Framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。