实时处理多模态数据流，用动态知识图谱提升响应速度与准确性。
StreamingRAG: Real-time Contextual Retrieval and Generation Framework
- 构建实时演化的多模态知识图谱，捕捉场景-对象-实体关系。
- 吞吐量提升5-6倍，资源消耗降低2-3倍，精度更高。
- 适合医疗、交通、遥感等需要实时分析的场景。
从医疗、智能交通、卫星遥感等多领域获取多模态数据流并提取实时洞察仍具挑战性。多模态大语言模型（MM-LLMs）面临高算力需求和知识范围有限的问题。传统检索增强生成（RAG）系统虽缓解知识局限，但预处理缓慢，难以满足实时分析需求。本文提出StreamingRAG，一种面向数据流的新颖RAG框架。该框架利用MM-LLMs构建随时间演化的知识图谱，实现对场景-对象-实体关系的实时捕捉，支持事件或查询的及时响应。相比现有方法，StreamingRAG在实时分析中实现5-6倍的吞吐量提升，通过时间感知知识图谱显著提高上下文准确性，并借助轻量级模型将资源消耗降低2-3倍。
原文摘要 · Abstract (English)
Extracting real-time insights from multi-modal data streams from various domains such as healthcare, intelligent transportation, and satellite remote sensing remains a challenge. High computational demands and limited knowledge scope restrict the applicability of Multi-Modal Large Language Models (MM-LLMs) on these data streams. Traditional Retrieval-Augmented Generation (RAG) systems address knowledge limitations of these models, but suffer from slow preprocessing, making them unsuitable for real-time analysis. We propose StreamingRAG, a novel RAG framework designed for streaming data. StreamingRAG constructs evolving knowledge graphs capturing scene-object-entity relationships in real-time. The knowledge graph achieves temporal-aware scene representations using MM-LLMs and enables timely responses for specific events or user queries. StreamingRAG addresses limitations in existing methods, achieving significant improvements in real-time analysis (5-6x faster throughput), contextual accuracy (through a temporal knowledge graph), and reduced resource consumption (using lightweight models by 2-3x).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。