arXiv:2605.14120cs.LGcs.CL2026-05

用小型专用模型舰队实现低成本精准水文智能推理

Mini-JEPA Foundation Model Fleet Enables Agentic Hydrologic Intelligence

论文配图:Mini-JEPA Foundation Model Fleet Enables Agentic Hydrologic Intelligence
图 1 · 摘自论文原文
  • 构建五个小型传感器专用模型,通过路由代理按需调用
  • 降水、温度、高程重建准确率分别达0.81、0.97、0.97
  • 适合需要低算力部署的水文监测与环境建模场景

地表基础模型将多光谱观测压缩为密集嵌入,广泛用于自然语言环境推理系统。单一全球尺度模型(如Google AlphaEarth)虽能泛化表征,但对专业水文信号表现有限,且往往难以获取、成本高昂、依赖大规模算力。本文提出Mini-JEPAs:一组小型传感器专用的联合嵌入预测架构(JEPA)基础模型,由路由代理根据问题选择合适模型。我们预训练了五个参数量2200万、共享同一视觉变换器主干、相同JEPA训练方案和64维输出空间的Mini-JEPAs,分别基于哨兵-2光学、哨兵-1 SAR、MODIS热红外、多时相哨兵-2物候及地形-土壤数据集。每个模型重建对应传感器变量,交叉验证的$R^2$达到:高程0.97,温度0.97,降水0.81。五个模型流形几何结构不同,全局参与度为8.9至20.2,局部内在维度为2.3至9.0。地形-土壤与物候模型组合在土壤湿度、干旱指数和降水预测上超越AlphaEarth单独使用,$ΔR^2$最高达0.031。路由大模型读取模态参考信息,对定制问题集实现完美命中率。在成对的LLM评判评估中,结合AlphaEarth与路由舰队的双重检索优于仅用AlphaEarth,在物理匹配问题上效果更优(Cohen's $d = 1.10$, $p = 0.031$)。本地训练的Mini-JEPAs可在低算力条件下实现水文智能应用。

原文摘要 · Abstract (English)

Geospatial foundation models compress multispectral observations into dense embeddings increasingly used in natural-language environmental reasoning systems. A single planetary-scale model, e.g. Google AlphaEarth, handles broad characterization well but may compromise on specialized hydrologic signals. Such generalist models are also often inaccessible, expensive, and require large-scale compute. We propose Mini-JEPAs: a fleet of small sensor-specialized Joint Embedding Predictive Architecture (JEPA) foundation models consulted by a routing agent for specialized questions. We pretrained five 22M-parameter Mini-JEPAs sharing an identical Vision Transformer backbone, JEPA recipe, and 64-d output space, using Sentinel-2 optical, Sentinel-1 SAR, MODIS thermal, multi-temporal Sentinel-2 phenology, and a topography-soil stack. Each Mini-JEPA reconstructs the variable matched to its sensor, with cross-validated $R^2$ reaching 0.97 for elevation, 0.97 for temperature, and 0.81 for precipitation. The five manifolds differ in geometric structure, with global participation ratios from 8.9 to 20.2 and local intrinsic dimensionalities from 2.3 to 9.0. Joint topography-soil and phenology models add predictive value beyond AlphaEarth alone for soil moisture, aridity, and precipitation ($ΔR^2$ up to 0.031). A router LLM reads per-modality references and selects appropriate sensors with a perfect hit rate over a curated question set. In paired LLM-as-Judge evaluation, dual retrieval over AlphaEarth and the routed fleet outperforms AlphaEarth alone on physics-matched questions (Cohen's $d = 1.10$, $p = 0.031$). Locally-trained Mini-JEPAs can be operationalized for hydrologic intelligence with modest compute.

水文智能小模型舰队地理模型路由代理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。