为超算环境设计分布式空间建模方法,提升预测连续性与精度。
A Partitioned Sparse Variational Gaussian Process for Fast, Distributed Spatial Modeling
- 将空间分区独立训练的稀疏变分高斯过程扩展为允许邻区轻量通信
- 在保持高可扩展性的前提下,使预测结果更平滑,整体拟合效果更好
- 适合超算中大规模分布式数据的实时建模,尤其适用于地球系统模拟
下一代能源部超算将具备百亿亿次计算能力,但数据存储能力无法跟上计算速度,用户将难以依赖事后数据分析进行不确定性量化等统计推断,亟需可在计算过程中就地训练的高效机器学习算法。这类算法必须高度可扩展、内存高效,并能处理跨节点分布的空间数据。一种可行方案是将稀疏变分高斯过程(SVGP)模型独立且并行地拟合到每个空间分区,该方法虽具可扩展性、高效性且通常准确,但导致相邻分区模型在边界处不一致,产生不连续的响应面。本文提出一种分区稀疏变分高斯过程(PSVGP),通过在邻近分区间引入少量通信,促进局部模型在边界上的对齐,从而实现更平滑的空间预测和更好的整体拟合。由于采用去中心化通信机制,该方法仍保持高度可扩展性,计算开销极低(内存无新增)。我们在能源百亿亿级地球系统模型(E3SM)上验证了该方法,并与独立训练的SVGP进行了对比。
原文摘要 · Abstract (English)
The next generation of Department of Energy supercomputers will be capable of exascale computation. For these machines, far more computation will be possible than that which can be saved to disk. As a result, users will be unable to rely on post-hoc access to data for uncertainty quantification and other statistical analyses and there will be an urgent need for sophisticated machine learning algorithms which can be trained in situ. Algorithms deployed in this setting must be highly scalable, memory efficient and capable of handling data which is distributed across nodes as spatially contiguous partitions. One suitable approach involves fitting a sparse variational Gaussian process (SVGP) model independently and in parallel to each spatial partition. The resulting model is scalable, efficient and generally accurate, but produces the undesirable effect of constructing discontinuous response surfaces due to the disagreement between neighboring models at their shared boundary. In this paper, we extend this idea by allowing for a small amount of communication between neighboring spatial partitions which encourages better alignment of the local models, leading to smoother spatial predictions and a better fit in general. Due to our decentralized communication scheme, the proposed extension remains highly scalable and adds very little overhead in terms of computation (and none, in terms of memory). We demonstrate this Partitioned SVGP (PSVGP) approach for the Energy Exascale Earth System Model (E3SM) and compare the results to the independent SVGP case.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。