让机器人团队主动选最合适的成员去查看被遮挡区域,提升户外协同感知的准确性和速度。
Co-GLANCE: Uncertainty-Aware Active Perception for Heterogeneous Robot Teaming

- 将视觉语言模型的语义推理能力蒸馏到轻量级模型,实现机载实时推理
- 通过校准的不确定性估计,使系统在遮挡分割和任务分配上分别提升25%和36%准确率
- 支持真实场景中多类型机器人协同,适合复杂环境下的自主团队作业
在非结构化室外环境中,异构机器人团队面临感知不确定性的核心挑战,单一视角难以获得可靠的场景理解。感知不确定性源于遮挡等因素,且在不同机器人视角下表现各异。检测与解决这些不确定性需要结合场景上下文推理和机器人能力匹配。虽然视觉语言模型具备强大的语义先验,但其计算开销大,不适用于机载推理,且缺乏校准的不确定性量化。我们提出 Co-GLANCE,一个面向异构机器人团队的实时机载感知与决策系统,用于不确定性消解。Co-GLANCE 将视觉语言模型的语义推理能力蒸馏为端到端的遮挡分割与机器人分配模型,无需云端推理。为量化感知不确定性,系统融合置信区间预测与选择性拒答,为分割、机器人分配和检测输出提供统计有效的覆盖保证。这些校准后的不确定性估计直接触发主动感知,派遣最合适的机器人获取信息丰富的视角以消除不确定性。在真实场景中,Co-GLANCE 在遮挡分割和机器人分配准确率上分别比云端视觉语言模型基线高出25%和36%,同时每帧推理延迟降低350倍。我们还发布了一个空地协同数据集供未来研究。代码、视频和数据集见 https://co-glance.github.io/
原文摘要 · Abstract (English)
Perceptual uncertainty is a central challenge for heterogeneous robot teams operating in unstructured outdoor environments, where no single viewpoint affords reliable scene understanding. Perceptual uncertainty, arising from sources such as occlusions, manifests differently across robot viewpoints depending on scene structure. Detecting and resolving sources of perceptual uncertainty requires both scene-based contextual reasoning and capability-aware robot allocation. While vision-language models provide strong semantic priors for both, they are computationally prohibitive for onboard inference and lack calibrated uncertainty quantification. We introduce Co-GLANCE, a real-time onboard perception and decision-making system for uncertainty resolution in heterogeneous robot teams. Co-GLANCE distills the semantic reasoning capabilities of a vision-language model into an end-to-end model for occlusion segmentation and robot allocation, eliminating the need for cloud-based inference. To quantify perceptual uncertainty, Co-GLANCE combines conformal prediction with selective abstention to provide statistically valid coverage guarantees for segmentation, robot allocation, and detection outputs. These calibrated uncertainty estimates directly trigger active perception, dispatching the most appropriate robot to acquire informative viewpoints and resolve uncertainty. Across real-world scenarios, Co-GLANCE outperforms cloud-based vision-language model baselines in occlusion segmentation and robot allocation accuracy by 25% and 36%, respectively, while reducing per-frame inference latency 350x. We also release an air-ground dataset for future research. Code, videos, and dataset available at https://co-glance.github.io/ .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。