为无人机搜救任务设计认知边界，防止大模型决策出错。
Cognition Envelopes for Bounded Decision Making in Autonomous UAS Operations
- 用概率推理和资源分析构建动态认知边界
- 在搜救任务中验证了决策准确率提升
- 适合高风险自主系统安全设计者参考
随着自主无人航空系统（UAS）越来越多依赖大型语言模型（LLMs）和视觉-语言模型（VLMs）实现感知、推理与规划，其产生的幻觉、过度泛化和上下文错配等问题也带来错误决策风险。为此，本文提出「认知包裹」（Cognition Envelopes）概念，通过设定推理边界来约束AI决策，同时结合元认知与传统安全边界。文中设计了一个基于LLM/VLM的线索分析流水线，用于小型自主无人机在搜救（SAR）任务中的动态判断，并建立以概率推理与资源分析为基础的认知包裹。通过多轮搜救任务评估该方法的有效性，结果显示决策质量显著改善。最后，识别出系统化设计、实现与验证认知包裹的关键软件工程挑战。
原文摘要 · Abstract (English)
Cyber-physical systems increasingly rely on foundational models, such as Large Language Models (LLMs) and Vision-Language Models (VLMs) to increase autonomy through enhanced perception, inference, and planning. However, these models also introduce new types of errors, such as hallucinations, over-generalizations, and context misalignments, resulting in incorrect and flawed decisions. To address this, we introduce the concept of Cognition Envelopes, designed to establish reasoning boundaries that constrain AI-generated decisions while complementing the use of meta-cognition and traditional safety envelopes. As with safety envelopes, Cognition Envelopes require practical guidelines and systematic processes for their definition, validation, and assurance. In this paper we describe an LLM/VLM-supported pipeline for dynamic clue analysis within the domain of small autonomous Uncrewed Aerial Systems deployed on Search and Rescue (SAR) missions, and a Cognition Envelope based on probabilistic reasoning and resource analysis. We evaluate the approach through assessing decisions made by our Clue Analysis Pipeline in a series of SAR missions. Finally, we identify key software engineering challenges for systematically designing, implementing, and validating Cognition Envelopes for AI-supported decisions in cyber-physical systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。