用动态经验驱动的多模态AI系统,提升灾害响应决策效率
RAPTOR-AI for Disaster OODA Loop: Hierarchical Multimodal RAG with Experience-Driven Agentic Decision-Making
- 构建分层多模态知识树,融合文本、图像与历史数据
- 检索精度提升23%,情境理解准确率提高31%
- 适合应急指挥、救援团队快速决策使用
人道主义援助与救灾(HADR)行动需在极端不确定性下快速整合多模态信息以支持关键决策。传统信息系统难以应对灾害数据碎片化、多源异构的问题,且缺乏动态推理能力。本文提出RAPTOR-AI,一种基于经验驱动的代理式多模态检索增强生成框架,在观察-定向-决策-行动(OODA)循环中实现动态决策支持。系统通过从46份海啸相关PDF(共2,378页)构建分层知识树,结合BLIP图像理解、ColVBERT嵌入与长文本摘要技术,实现多模态知识融合。引入熵感知的代理控制机制,根据情境动态选择最优检索策略,并利用LoRA微调整合专家与非专家经验。实验表明,相比现有方法,检索精度提升23%,情境认知准确率提高31%,任务分解准确率提升27%,可高效扩展至3,000个文档块。
原文摘要 · Abstract (English)
Humanitarian Assistance and Disaster Relief (HADR) operations demand rapid synthesis of multimodal information for time-critical decision-making under extreme uncertainty. Traditional information systems struggle with the fragmented, multimodal nature of disaster data and lack adaptive reasoning capabilities essential for dynamic emergency contexts. This work introduces RAPTOR-AI, an agentic multimodal Retrieval-Augmented Generation (RAG) framework that advances beyond conventional static knowledge bases by implementing dynamic, experience-driven decision support for disaster response. The system addresses HADR requirements across initial rescue, recovery, and reconstruction phases through three key innovations: hierarchical multimodal knowledge construction from diverse sources (textual reports, aerial imagery, historical documentation), entropy-aware agentic control that dynamically selects optimal retrieval strategies based on situational context, and experiential knowledge integration using LoRA adaptation for both expert and non-expert responders. The framework constructs hierarchical knowledge trees from 46 tsunami-related PDFs (2,378 pages) using BLIP-based image understanding, ColVBERT embeddings, and long-context summarization within the OODA loop (Observe, Orient, Decide, Act) tactical framework. Experiments demonstrate significant improvements over existing approaches: 23\% improvement in retrieval precision, 31\% better situational grounding, and 27\% enhanced task decomposition accuracy, with efficient scaling up to 3,000 document chunks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。