通过星地协作让卫星实时运行大模型，大幅降低遥感延迟
Enabling Near-realtime Remote Sensing via Satellite-Ground Collaboration of Large Vision-Language Models
- 星上部署轻量模型，地面用大模型协同推理，分担计算压力
- 实测显示平均延迟降低76%-95%，精度不下降
- 适合需要快速响应的灾害监测等实时遥感场景
大型视觉语言模型（LVLM）在低地球轨道（LEO）卫星的遥感任务（如灾害监测）中展现出巨大潜力。然而，受限于星上计算资源不足和星地通信时间短，其在真实卫星系统中的部署仍鲜有探索。本文提出Grace系统，一种面向遥感任务的近实时LVLM推理星地协同架构。系统采用异步星地检索增强生成（RAG）机制，并设计任务调度算法：星上部署轻量级LVLM实现实时推理，地面站则运行更大模型以保障端到端性能。在有限的星地通信窗口内，通过自适应更新算法将地面知识库同步至卫星；同时提出基于置信度的判断策略，决定任务是否在星上处理或卸载至地面。基于真实卫星轨道数据的大量实验表明，Grace相比现有最优方法，平均延迟降低76%至95%，且不牺牲推理准确率。
原文摘要 · Abstract (English)
Large vision-language models (LVLMs) have recently demonstrated great potential in remote sensing (RS) tasks (e.g., disaster monitoring) conducted by low Earth orbit (LEO) satellites. However, their deployment in real-world LEO satellite systems remains largely unexplored, hindered by limited onboard computing resources and brief satellite-ground contacts. We propose Grace, a satellite-ground collaborative system designed for near-realtime LVLM inference in RS tasks. Accordingly, we deploy compact LVLM on satellites for realtime inference, but larger ones on ground stations (GSs) to guarantee end-to-end performance. Grace is comprised of two main phases that are asynchronous satellite-GS Retrieval-Augmented Generation (RAG), and a task dispatch algorithm. Firstly, we still the knowledge archive of GS RAG to satellite archive with tailored adaptive update algorithm during limited satellite-ground data exchange period. Secondly, propose a confidence-based test algorithm that either processes the task onboard the satellite or offloads it to the GS. Extensive experiments based on real-world satellite orbital data show that Grace reduces the average latency by 76-95% compared to state-of-the-art methods, without compromising inference accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。