测试视觉语言模型跨摄像头推理能力,发现现有模型表现不佳。
CrossView: Can Vision-Language Models Reason Across Cameras?

- 构建多视角视频问答数据集,模拟真实多摄像头场景。
- 主流模型在跨视角任务中准确率普遍低于40%,开源模型差距更大。
- 适合研究多模态感知、自动驾驶与机器人视觉的学者使用。
视频理解基准长期聚焦单摄像头设置,现代多模态语言模型在此类任务中表现优异。然而现实世界依赖多摄像头网络:自动驾驶、安防系统和机器人均需同步处理多个视角数据。我们认为这并非单摄像头问题的简单扩展,而是本质不同:多摄像头推理需应对视角数量增长带来的上下文复杂性,解决仅部分摄像头可见的遮挡问题,判断哪些视角关键,并整合可能重叠或分歧的多视角证据。当前模型正面临这些挑战,但尚无系统性评测基准。为此,我们提出CrossView,一个涵盖自动驾驶、安防监控、第一人称/第三人称视频及机器人场景的多摄像头视频问答基准。对私有模型(如GPT-5.2)和开源模型(如Qwen3-VL)的评估显示,准确率普遍偏低,开源模型差距显著。性能与模型联合处理多视角的能力强相关,验证了CrossView作为严谨多摄像头视频理解评测基准的有效性。代码与数据集已开源:https://utaustin-swarmlab.github.io/CrossView。
原文摘要 · Abstract (English)
Video understanding benchmarks have long centered on single-camera settings, where modern multi-modal language models achieve strong performance across image and video tasks. Yet, the real world runs on multi-camera networks: autonomous vehicles, security systems, and robots all gather data across many simultaneous views. We argue that this is not simply "more" of the single-camera problem; it is fundamentally different. Multi-camera reasoning requires handling context that scales with the number of views, resolving occlusions visible from only a subset of cameras, judging which views matter, and integrating evidence across perspectives that may overlap or diverge. Current models struggle with exactly these challenges, yet no benchmark systematically targets them. We introduce CrossView, a multi-camera video question-answering benchmark spanning autonomous driving, security surveillance, egocentric/exocentric video, and robotics. Evaluation of proprietary models, such as GPT-5.2, and open-source models, like Qwen3-VL, reveals consistently low accuracy, with open-source models trailing by a wide margin. Performance scales strongly with a model's ability to jointly process multiple viewpoints, positioning CrossView as a rigorous benchmark for multi-camera video. We open-source our code and dataset at https://utaustin-swarmlab.github.io/CrossView.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。