用大模型整合物理先验，提升遥感图像分割精度与合理性
Transferring Physical Priors into Remote Sensing Segmentation via Large Language Models
- 通过大模型构建物理知识图谱，融合多源地理物理数据
- 新模型PriorSeg在无重训练下提升分割准确率与物理合理性
- 适合遥感、地球观测领域研究者，尤其关注多模态数据融合
遥感图像语义分割是地球观测的基础。实现高精度分割需融合光学影像及数字高程模型（DEM）、合成孔径雷达（SAR）和归一化植被指数（NDVI）等物理变量。现有基础模型虽能利用这些变量，但仍依赖空间对齐数据，并在引入新传感器时需昂贵的重新训练。为此，我们提出一种将领域特定物理先验融入分割模型的新范式。首先，通过提示大语言模型从1,763个术语中提取物理先验，构建以物理为中心的知识图谱（PCKG），并基于此生成异构且空间对齐的数据集Phy-Sky-SA。在此基础上，开发了物理感知的残差精修模型PriorSeg，采用视觉-物理联合训练策略，引入新型物理一致性损失。在异构设置下的实验表明，PriorSeg在不重训练基础模型的前提下，提升了分割准确率与物理合理性。消融实验证明了Phy-Sky-SA数据集、PCKG及物理一致性损失的有效性。
原文摘要 · Abstract (English)
Semantic segmentation of remote sensing imagery is fundamental to Earth observation. Achieving accurate results requires integrating not only optical images but also physical variables such as the Digital Elevation Model (DEM), Synthetic Aperture Radar (SAR) and Normalized Difference Vegetation Index (NDVI). Recent foundation models (FMs) leverage pre-training to exploit these variables but still depend on spatially aligned data and costly retraining when involving new sensors. To overcome these limitations, we introduce a novel paradigm for integrating domain-specific physical priors into segmentation models. We first construct a Physical-Centric Knowledge Graph (PCKG) by prompting large language models to extract physical priors from 1,763 vocabularies, and use it to build a heterogeneous, spatial-aligned dataset, Phy-Sky-SA. Building on this foundation, we develop PriorSeg, a physics-aware residual refinement model trained with a joint visual-physical strategy that incorporates a novel physics-consistency loss. Experiments on heterogeneous settings demonstrate that PriorSeg improves segmentation accuracy and physical plausibility without retraining the FMs. Ablation studies verify the effectiveness of the Phy-Sky-SA dataset, the PCKG, and the physics-consistency loss.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。