用大模型实现冰湖分割与空间推理，让机器像人一样解释位置关系。
GLACIA: Instance-Aware Positional Reasoning for Glacial Lake Segmentation via Multimodal Large Language Model
- 融合大语言模型与分割网络，实现像素级预测+自然语言推理
- 在冰湖数据集上达到87.30的mIoU，显著优于传统方法
- 适合需要可解释性决策的冰川灾害预警场景
冰湖监测对防范冰湖溃决洪水具有重要意义。然而，基于卷积神经网络（CNN）和视觉变换器（ViT）的现有分割方法仍局限于像素级预测，缺乏高层次全局场景语义和人类可理解的推理能力。为此，我们提出GLACIA（Glacial Lake segmentation with Contextual Instance Awareness），首个将大语言模型与分割能力结合的框架，既能生成精确的分割掩码，又能输出对应的空间推理结果。我们构建了冰湖位置推理（GLake-Pos）数据集流水线，提供多样且空间定位明确的问答对，以弥补遥感领域实例感知位置推理数据的不足。对比实验表明，GLACIA在mIoU上达到87.30，显著超越基于CNN的方法（78.55–79.01）、ViT（69.27–81.75）、地理基础模型（76.37–87.10）以及基于推理的分割方法（60.12–75.66）。该方法通过自然语言交互，支持直观的灾害预判与科学决策，提升在快速变化冰川环境中的响应效率。代码已开源：https://github.com/lalitmaurya47/GLACIA
原文摘要 · Abstract (English)
Glacial lake monitoring bears great significance in mitigating the anticipated risk of Glacial Lake Outburst Floods. However, existing segmentation methods based on convolutional neural networks (CNNs) and Vision Transformers (ViTs), remain constrained to pixel-level predictions, lacking high-level global scene semantics and human-interpretable reasoning. To address this, we introduce GLACIA (\textbf{G}lacial \textbf{LA}ke segmentation with \textbf{C}ontextual \textbf{I}nstance \textbf{A}wareness), the first framework that integrates large language models with segmentation capabilities to produce both accurate segmentation masks and corresponding spatial reasoning outputs. We construct the Glacial Lake Position Reasoning (GLake-Pos) dataset pipeline, which provides diverse, spatially grounded question-answer pairs designed to overcome the lack of instance-aware positional reasoning data in remote sensing. Comparative evaluation demonstrate that GLACIA (mIoU: 87.30) surpasses state-of-the-art method based on CNNs (mIoU: 78.55 - 79.01), ViTs (mIoU: 69.27 - 81.75), Geo-foundation models (mIoU: 76.37 - 87.10), and reasoning based segmentation methods (mIoU: 60.12 - 75.66). Our approach enables intuitive disaster preparedness and informed policy-making in the context of rapidly changing glacial environments by facilitating natural language interaction, thereby supporting more efficient and interpretable decision-making. The code is released on https://github.com/lalitmaurya47/GLACIA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。